Back to Feed
Computer Vision / Multimodal

Transferring Motion Between Different Looking Objects

Original: Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations
By Zhixue Fang, Zhimin Zhang, Bi'an Du, Zijie Meng, Yan Zhou, Wei Hu, Guoxin Zhang, Pengfei Wan, Kun Gai Published 2026-08-03

Listen to the summary

Uses a voice available on your device

Audio options
On this page

Key Takeaways

  • Most existing tools struggle to transfer movement between objects with different shapes, but this new approach bridges that gap.
  • The method uses a two-stage process that first teaches the system to understand basic movement patterns and then applies that knowledge to real videos.
  • The new technique showed high performance in human tests, being preferred over existing methods in 93 percent of image-to-video tests and 97 percent of text-to-video tests.
  • By moving away from explicit motion extraction during final use, the system becomes more versatile at handling diverse types of visual content.

Summary & Methodology Analysis

The paper tackles a core problem in video generation where systems usually require objects to have similar body shapes or structures to transfer motion effectively. To solve this, the authors introduce a two-stage training strategy. In the first stage, they perform abstract motion bootstrapping, which means the model learns essential movement patterns like trajectories (the path an object takes) and semantic kinematics (the study of how parts move in relation to one another) through various types of data views. This phase also involves creating a new collection of video pairs that represent equivalent movements across different object categories to train the system effectively.

Interactive System Flowchart

Click diagram to expand and zoom

Cross-Examination & FAQs

A deeper dive clarifying mechanics, constraints, and baseline evaluations.

Q1. What is the main problem this research solves?

Current video movement tools fail when trying to transfer motion between objects that look different or have different body structures.

Q2. Does this method require the objects to be similar?

No, the method is specifically designed to work across different categories where structural correspondence is absent or weak.

Q3. How successful was the method in human tests?

It was highly successful, winning preference in 93.0 percent of image-to-video tests and 97.3 percent of text-to-video tests.

Q4. What happens in the second stage of training?

In the second stage, the model internalizes the motion, replacing abstract conditions with raw reference videos to learn how to infer dynamics directly.

Q5. Is explicit motion extraction required during the final use of the model?

No, the final inference process allows the generator to be conditioned directly on a reference video and a target prompt without needing to extract motion explicitly.

Q6. What are the limitations of the motion abstractions used in the first stage?

The individual motion abstractions have limited information capacity and coverage, which restricts their detail.

Q7. Are there challenges in identifying which movements can be transferred?

Yes, while the method assumes some dynamics are meaningful across categories, it is difficult to identify transferable factors for certain pairs of objects.

Q8. What datasets and benchmarks were involved in the paper?

The paper utilized several tools including the OpenVMT-Dataset, OpenVMT-Bench, VBench, VBench++, MotionBench, MTBench, DisMo, Wan-Move, Tora, DeT, FlowMotion, Seedance-2.0, and various tools like ViTPose++ and CoTracker2.

Q9. Does the paper specify the hardware required for training?

The paper does not specify the hardware requirements for training or inference.