Personalizing Images Without Paired Data

Summary & Methodology Analysis

Subject-driven image personalization traditionally relies on costly multi-stage curation pipelines to generate large-scale paired reference and composed target data for training. CRAFT addresses this operational bottleneck by enabling personalization with only reference-side supervision. The method profiles the frozen MMDiT backbone, a type of transformer architecture designed for multimodal generation, to select reward locus coordinates, specifically the denoising step and transformer blocks, where attention best aligns with reference subjects. It then implements the where-to-look principle using attention-level rewards, including noise-to-reference, phrase-to-reference, and spatial consistency, to align routing within the transformer. Per-subject attention masks are derived from the noise-to-reference attention maps to localize identity supervision, and a pixel-level identity reward is gated using these masks to ensure image-space supervision remains consistent with learned attention routing. Finally, LoRA adapters, which are lightweight parameter modules used for efficient fine-tuning, are updated via single-step Reward Feedback Learning, a reinforcement learning technique, using the combined attention-level and gated pixel-level rewards.

In evaluations, CRAFT achieved an overall score of 76.47 on the XVerseBench benchmark using only 10,000 reference-only instances, outperforming methods requiring 150,000 to over 2 million composed-target pairs. On the single-subject split of XVerseBench, the approach obtained a state-of-the-art average score of 81.62. The models and datasets referenced in the paper include FLUX.2-klein-9B, XVerseBench, DreamBench, OmniContext, DreamBench++, UNO, and UMO. The paper does not specify precise hardware specifications, latency numbers, or deployment costs.

Despite strong single-subject performance, the framework has notable limitations. CRAFT's performance advantage is less pronounced on multi-subject scenes compared to single-subject prompts, with multi-subject identity scores trailing the strongest composed-target baseline. Furthermore, the framework requires a backbone that natively supports reference tokens, and performance is constrained by the base model's capacity since the method amplifies existing routing rather than introducing new structural capabilities.