Key takeaways
- Persist the edit mask and protected regions so reviewers know exactly which pixels a model was allowed to replace.
- Use documented /image/generate inpainting for mask-based synthesis and deterministic compositing when the replacement asset must remain exact.
- Never overwrite the approved source; treat generated replacements as reviewable candidates with provenance.
AI replacement can make a complex retouch feel like one prompt, but selection and review still determine quality. Small mask errors can change hands, text, edges, reflections, and objects that were never intended to move.
Choose synthesis or exact compositing deliberately
Replacement can mean pasting an approved subject, generating new pixels inside a region, or rebuilding a background after removal. Exact compositing is appropriate when the inserted object or text must remain unchanged. Inpainting is appropriate only where plausible synthesis is allowed and reviewable.
Define the protected source regions, desired replacement, lighting and perspective constraints, output placements, and deterministic fallback. Do not let the most convenient model choose whether evidence, product details, identity, or branded text may change.
Create source-aligned masks and protected regions
Build the mask against the exact normalized source and store width, height, checksum, orientation, and mask convention. Feathering can help visual blending but can also expose protected pixels to modification. Validate alignment with an overlay before sending the request.
Keep separate regions for editable pixels, protected subject details, and contextual review. If the source is resized or rotated, transform or regenerate the mask deliberately. Never reuse a mask after source replacement merely because dimensions happen to match.
Use documented image-and-mask inpainting roles
The Assembly below gives two uploads semantic image and mask roles for /image/generate with an inpainting model. Name upload fields semantically or use as directives, and describe only the intended masked change. The response remains a candidate, not an approved edit.
Validate file type, dimensions, and decodability, then compare protected regions with the source. Store model, prompt version, source, mask, and provider job evidence. A successful request does not establish exact preservation or visual consistency.
{
"steps": {
":original": { "robot": "/upload/handle" },
"inpainted": {
"use": [
{ "name": ":original", "as": "image" },
{ "name": ":original", "as": "mask" }
],
"robot": "/image/generate",
"model": "stability-ai/stable-diffusion-inpainting",
"prompt": "Replace only the masked area with the approved subject. Keep the rest of the image unchanged.",
"format": "png"
}
}
}Retain source, mask, candidate, and approved output
Model output, reviewer correction, and final delivery derivative are different records. Keep the original immutable and preserve the candidate even when an editor creates a corrected version under policy. This history supports rollback and explains which changes were machine-generated or human-approved.
Run deterministic resize, optimization, and export after approval or keep those outputs quarantined. Separate stage retries: storage failure should not rerun inpainting, and inference failure should not destroy the source or prior approved asset.
Inspect boundaries, light, text, and physical logic
Review mask edges, occlusion, shadows, reflections, scale, perspective, texture repetition, anatomy, text, logos, and causal consistency. Inspect both the master and each target crop. Automated pixel comparisons can protect unchanged areas, while people assess plausibility and meaning.
Use real edit briefs and negative cases that should choose compositing or no change. Track acceptance, correction time, protected-region difference, and rejection reason. A high aesthetic score cannot compensate for altered evidence or a replacement that violates the stated brief.
Revise the mask before expanding model freedom
When a candidate fails at a boundary, inspect mask alignment and protected regions before broadening the prompt or editable area. A larger mask may conceal the immediate seam while allowing more unintended changes. Prefer a tighter technical correction or exact compositing when control matters.
Cap retries and make refusal, timeout, malformed output, failed preservation checks, and reviewer rejection separate terminal states. Keep user-visible edits recoverable and never allow a late candidate to overwrite a newer source or decision.
Prevent stale edits from following a changed source
Version source, mask, prompt, model, review rubric, and finishing workflow together. Before accepting a callback or publishing a candidate, compare its source checksum and edit generation with the active record. Mark mismatched outputs stale rather than attempting to transplant them.
Monitor protected-pixel differences, rejection reasons, retries, correction, provider drift, latency, and cost by edit type. Re-run fixed source-and-mask pairs after provider changes and keep the prior approved output available until the replacement passes review.
Technical details worth knowing
- Task boundary: AI image replacement uses a mask and instructions to synthesize a new object or region inside an existing picture. Inpainting synthesizes pixels inside a mask; compositing places known source pixels, while retouching may use deterministic local edits.
- Input contract: Provide a precise source-aligned mask, protected regions, replacement constraints, and references while preserving which pixels were original. Input preparation must be evaluated with the model because preprocessing can remove evidence as well as noise.
- Output contract: Produce a candidate plus source-aligned edit mask, prompt and model provenance, protected-region record, review state, and approved derivative set. A valid response does not prove that a predicted label, region, or generated pixel is correct.
- Method choice: Use /image/generate inpainting with an explicitly supported model when synthesis is acceptable and the output can receive visual and factual review; prefer deterministic compositing when a known replacement asset must remain exact. Model names alone do not describe the training data, thresholds, latency, licensing, or failure behavior of a deployed system.
- Evaluation: Review mask boundaries, lighting, perspective, repeated patterns, text and logo integrity, physical plausibility, source fidelity, and correction time. Aggregate scores should be segmented by content type so common easy examples do not hide failures on important edge cases.
- Failure and safety: A failed replacement must leave the approved source untouched and offer mask revision, deterministic compositing, or human editing as an explicit fallback. Disclose material edits where context requires it and review replacements involving people, evidence, products, trademarks, or regulated claims.
- Operations: Store masks and protected regions, version prompts and models, monitor rejection reasons, and prevent revised source assets from inheriting stale edits.
A practical approach
- 1
Write the decision, output schema, and rejection criteria for mask-based AI image replacement.
- 2
Build a representative mask-based AI image replacement evaluation set and preserve each source, preprocessing choice, and provenance record.
- 3
Benchmark the complete workflow on representative evidence and compare the result with predefined task-specific acceptance criteria.
- 4
Release mask-based AI image replacement behind explicit review and fallback paths, then monitor the operating signals that determine whether it remains useful.
When Transloadit is useful
Use /image/generate with two separate uploads—a source image and mask—tagged through semantic field names or the as directives. Use /image/resize for approved derivatives and storage Robots to retain the source, mask, candidate, and reviewed final.
Architecture boundary
Transloadit /image/generate documents image-and-mask inpainting, including explicit image and mask roles and supported options such as stability-ai/stable-diffusion-inpainting. Results remain model-dependent; use a specialist only when the documented models or contract do not cover the required edit.
Frequently asked questions
Is inpainting the same as copying an exact replacement into an image?
No. Inpainting synthesizes pixels under model behavior. Use deterministic compositing when the inserted object, logo, text, or product details must remain exact.
Why must the mask be tied to a source checksum?
Mask coordinates describe one exact image. A changed crop, orientation, resize, or source can make the same-looking mask edit the wrong pixels.
Should a failed edge lead to a larger mask?
Not automatically. First inspect alignment, polarity, feathering, and protected regions. Expanding the mask increases the area where the model may introduce unintended changes.
What should be compared automatically after inpainting?
Validate files and dimensions, assert source and mask identity, and compare protected regions under an appropriate tolerance. Human review still determines visual and semantic acceptability.