Key takeaways
- Persist crop geometry in normalized source coordinates so every output can be regenerated consistently.
- Choose the detector around the real subject—face, product, text, or composition—not a generic “smart crop” label.
- Use padding or human art direction when no automated crop can preserve the required information.
AI can suggest a crop, but the application still owns what must remain visible. Product labels, gestures, negative space, and editorial meaning routinely matter more than the model’s most visually salient region.
Name what every placement must keep visible
Turn each destination into a crop contract: target ratio, minimum output size, required subjects, protected text or logos, allowed padding, and whether a reviewer must approve it. A crop for a product card, portrait thumbnail, and editorial hero can use the same source but have incompatible success criteria.
This guide focuses on evaluating crop proposals across aspect ratios, rather than the broader architecture covered by the automatic-cropping guide. Define failure in visible terms such as clipped faces, missing product features, broken text, lost context, or a subject reduced below a usable size.
Normalize orientation and source coordinates
Apply orientation consistently before a detector or cropper sees the image, and define all stored geometry against that normalized source. Record source width and height with rectangles or focal points. Otherwise a later rotation, resize, or replacement can make apparently valid coordinates select the wrong region.
Prefer normalized coordinates for reusable intent, but retain the precise source version and rounding rules. Test asymmetric images that expose x/y reversal, edge-touching subjects, odd dimensions, and one-pixel boundaries. Never reuse crop geometry after the source pixels change without revalidation.
Choose face, object, saliency, or human art direction
Face detection is useful when a face is truly the protected subject; an object detector is more appropriate for products or vehicles. Attention and entropy heuristics can propose visually active regions but do not understand editorial meaning. Human focal intent remains the strongest input for brand-critical compositions.
Compare candidates on the same source set instead of declaring one signal universally smart. Include group portraits, small subjects, text-heavy graphics, products with accessories, and images whose important context is visually quiet. Record which signal produced each proposal so failures remain attributable.
Render approved crops as reproducible derivatives
The example creates one 1200×630 candidate with attention gravity. It is a deterministic proposal, not proof that the important subject survived. Store the exact dimensions, resize strategy, gravity or crop coordinates, source checksum, and application-managed workflow version before review.
Generate each approved destination from the same source rather than repeatedly cropping a previous derivative. This avoids compounded rounding and lost context. Keep a link from every derivative to its crop decision so an editor can regenerate or replace the complete placement set safely.
{
"steps": {
":original": { "robot": "/upload/handle" },
"crop_candidate": {
"use": ":original",
"robot": "/image/resize",
"result": true,
"resize_strategy": "fillcrop",
"gravity": "attention",
"width": 1200,
"height": 630
}
}
}Score subject retention across aspect ratios
Evaluate per placement. Measure protected-region coverage, intersection with forbidden cut lines, subject scale, text completeness, and reviewer acceptance. A proposal that succeeds for a landscape hero may fail for a square card, so one accepted crop must not stand in for the whole destination matrix.
Use fixtures with subjects near every edge, multiple competing faces, small products, vertical text, transparent backgrounds, and no detectable subject. Track the number and severity of manual adjustments. Pixel comparison can verify deterministic reproduction, while human review evaluates composition and meaning.
Use padding when no honest crop exists
Some source-and-ratio combinations cannot preserve all required content. Detect that condition explicitly and choose a background pad, letterbox, alternate artwork, or review queue. Do not zoom until the subject becomes unusably small or cut required information merely to satisfy exact output dimensions.
Fallback order belongs in the placement contract. Keep the approved original unchanged when detection fails, cap retries, and show the editor why a fallback was selected. A missing crop should be visible workflow state, not an empty derivative that downstream publishing treats as success.
Learn from editor overrides without moving the source
Record editor changes as focal intent or approved crop geometry linked to the source version. Analyze overrides by placement and content class: repeated horizontal shifts may reveal a detector bias, while frequent padding may show that the requested ratio is incompatible with supplier imagery.
Re-run regression fixtures after changing a detector, threshold, resize stack, or rounding rule. Keep the prior workflow available during rollout and invalidate stored geometry when the source changes. Monitor failed crops, manual adjustment distance, review time, and destination-specific rejection.
Technical details worth knowing
- Task boundary: AI-assisted cropping proposes a source rectangle that preserves the subject while fitting a required output ratio. Crop selection chooses a meaningful rectangle; resizing changes dimensions, and generative expansion invents pixels when the desired frame is larger than the source.
- Input contract: Correct orientation before analysis and express the approved crop in normalized source coordinates so preview size and device pixel ratio do not change it. Input preparation must be evaluated with the model because preprocessing can remove evidence as well as noise.
- Output contract: Return normalized source coordinates, target aspect ratio, detected anchors, confidence, policy version, and review state rather than only a rendered bitmap. A valid response does not prove that a predicted label, region, or generated pixel is correct.
- Method choice: Compare /image/resize attention and entropy gravity with face detection for portraits, a specialist detector for known products, and manual art direction where composition carries editorial meaning. Model names alone do not describe the training data, thresholds, latency, licensing, or failure behavior of a deployed system.
- Evaluation: Score subject retention, face and text clipping, composition, consistency across aspect ratios, reviewer edits, and the downstream performance of accepted crops. Aggregate scores should be segmented by content type so common easy examples do not hide failures on important edge cases.
- Failure and safety: When no safe crop meets the target ratio, keep the source, request art direction, or use padding instead of forcing a destructive crop. Do not infer identity or protected traits from a detected face, and review crops that could change the apparent meaning of news, evidence, or sensitive imagery. /image/facedetect uses AWS or GCP, so evaluate provider egress, region, and retention before sending portraits.
- Operations: Store normalized coordinates and rationale, track manual adjustments, and regenerate derivatives from the source when aspect ratios or crop policy change.
A practical approach
- 1
Write the decision, output schema, and rejection criteria for subject-aware image cropping.
- 2
Build a representative subject-aware image cropping evaluation set and preserve each source, preprocessing choice, and provenance record.
- 3
Benchmark the complete workflow on representative evidence and compare the result with predefined task-specific acceptance criteria.
- 4
Release subject-aware image cropping behind explicit review and fallback paths, then monitor the operating signals that determine whether it remains useful.
When Transloadit is useful
Use /image/resize with explicit coordinates or its attention and entropy gravity modes as transparent baselines, and /image/facedetect when faces are the intended subject. Store the selected coordinates before rendering final derivatives.
Architecture boundary
Transloadit /image/resize supports deterministic crops plus attention- and entropy-based point-of-interest gravity, and /image/facedetect supports face-aware crops. These heuristics are not universal aesthetic or product-saliency judges, so consequential crops still need approval.
Frequently asked questions
Is attention gravity an AI subject detector?
No. It favors visual cues such as luminance frequency, saturation, and skin-tone signals. Treat it as one crop heuristic and evaluate it against the actual subjects and placements.
Can one crop rectangle serve every aspect ratio?
Rarely. Each ratio changes what can remain visible and how large the subject appears. Evaluate a placement matrix or store focal intent that each destination resolves under explicit rules.
What should happen when no crop preserves the required content?
Use the predefined padding, alternate-artwork, or review fallback. The system should report that no valid crop exists instead of silently clipping protected information.
How can crop regressions be tested?
Keep asymmetric fixtures with known protected regions and assert dimensions, coordinate transforms, and subject coverage. Add reviewer acceptance for composition, which deterministic pixel tests cannot establish.