Fix input validation in image resampling and transform pipelines

[sayanshaw24]

Fix input validation issues in image resampling and transform pipelines

Summary

This PR adds missing input validation to the image processing pipeline to prevent crashes and out-of-bounds memory access from malformed inputs.

Changes

shared/api/image_resample.c

• Fix integer overflow in precompute_coeffs(): the overflow-protection check itself used int arithmetic that could wrap on extreme resize ratios (e.g., 50M pixels → 1 pixel). Replaced with size_t arithmetic and added an early guard on support before computing ksize.
• Add box coordinate validation in ImagingResample(): reject reversed coordinates (left >= right or top >= bottom), out-of-image-bounds boxes, and non-positive output dimensions. Previously, reversed box values caused undersized buffer allocations leading to out-of-bounds reads.

shared/api/image_transforms.hpp

• Add NULL check on ImagingResample() return value in Resize::Compute(). The function can return NULL on error (unsupported mode, allocation failure, invalid parameters) but the result was dereferenced unconditionally.

shared/api/image_transforms_phi_4.hpp

• Add NULL check on ImagingResample() return value in the Phi-4 vision resize path (same issue as above).

shared/api/image_transforms_phi_3.hpp

• Add channel count validation in phi3_hd_transform(). The function reads the channel count from the input tensor shape but allocates the output buffer with a hardcoded 3-channel layout. A non-3-channel input would cause writes past the allocated buffer. This matches the existing validation already present in the Phi-4 transform.

Testing

These fixes are pure input-validation guards. The overflow path requires a ~50-million-pixel image, the NULL deref requires an allocation failure or unsupported mode (which upstream decoders already reject), and the channel mismatch requires bypassing the RGB decoder that hardcodes 3 channels. All are defense-in-depth checks.

[Copilot]

Pull request overview

This PR hardens the shared image preprocessing pipeline by adding defensive input validation around resampling and model-specific vision transforms, aiming to prevent OOB accesses and null dereferences when malformed inputs or allocation failures occur.

Changes:

• Tighten overflow protection in precompute_coeffs() and use size_t for allocation size math.
• Add output-size and crop-box validation in ImagingResample() to reject invalid resize requests.
• Add new null checks after ImagingResample() in the generic resize path and Phi-4 vision preprocessing, and add a channel-count validation for Phi-3 HD transform.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
shared/api/image_resample.c	Adds overflow/parameter validation in resampling, and adjusts allocation arithmetic.
shared/api/image_transforms.hpp	Adds a null check after resampling in Resize::Compute().
shared/api/image_transforms_phi_4.hpp	Adds a null check after resampling in Phi-4 dynamic preprocess resize path.
shared/api/image_transforms_phi_3.hpp	Adds channel-count validation to prevent output buffer overruns for non-RGB inputs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.