Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat: add CameraMotionCompensator to improve tracking performance with camera motion #2009

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
westlinkin wants to merge 2 commits into roboflow:develop
base: develop
Choose a base branch
Loading
from westlinkin:develop

Conversation

@westlinkin
Copy link

@westlinkin westlinkin commented Nov 6, 2025

Description

This PR introduces CameraMotionCompensator, a utility class that improves tracking performance with camera motion. This addresses the problem where camera movement (panning, shaking, etc.) degrades tracker accuracy, leading to ID switches and lost tracks.

No extra dependencies that are required for this change.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

The feature has been tested using a real-world video processing script. The following example demonstrates how to use the CameraMotionCompensator in "Simple Mode" within the sv.process_video callback structure. This confirms that the motion calculation is performed once per frame and correctly applied to the tracker.

import numpy as np
import cv2
import supervision as sv
from ultralytics import RTDETR
def process_video_with_cmc():
 # Setup models, video paths, etc.
 helmet_model = RTDETR(<HELMET_MODEL_PATH>)
 clip_path = <VIDEO_PATH>
 output_path = "result.mp4"
 # Initialize tools
 box_annotator = sv.BoxAnnotator()
 label_annotator = sv.LabelAnnotator()
 
 # Initialize tracker and wrap it with CameraMotionCompensator
 byte_tracker = sv.ByteTrack(frame_rate=<FRAME_RATE>)
 cmc = sv.CameraMotionCompensator(tracker=byte_tracker)
 cmc.reset()
 def callback(frame: np.ndarray, index: int) -> np.ndarray:
 # 1. Perform detection
 results = helmet_model(frame)[0]
 detections = sv.Detections.from_ultralytics(results)
 # 2. Update with detections using the simple wrapper method
 final_detections = cmc.update_with_detections(detections, frame=frame)
 # 3. Annotation
 labels = [
 f"#{tracker_id} {class_name} {confidence:.2f}"
 for class_name, tracker_id, confidence
 in zip(final_detections.data["class_name"], final_detections.tracker_id, final_detections.confidence)
 ]
 annotated_frame = box_annotator.annotate(frame.copy(), detections=final_detections)
 return label_annotator.annotate(annotated_frame, detections=final_detections, labels=labels)
 sv.process_video(
 source_path=clip_path,
 target_path=output_path,
 callback=callback
 )
if __name__ == "__main__":
 process_video_with_cmc()

Any specific deployment considerations

None

Docs

  • Docs updated? What were the changes:

@westlinkin westlinkin changed the title (削除) Add CameraMotionCompensator to improve tracking performance with camera motion (削除ここまで) (追記) feat: add CameraMotionCompensator to improve tracking performance with camera motion (追記ここまで) Nov 6, 2025
Copy link
Collaborator

SkalskiP commented Nov 6, 2025

Hi @westlinkin 👋🏻 Thanks a lot for your interest in Supervision. Could you share a Google Colab notebook showing how CMC is used in practice? It would help me review this PR faster.

Copy link
Author

I cannot share my custom trained model, but here is the results of without cmc video and cmc video:

No. 4 is missed in no-cmc version at the end of the video

Here is the colab notebook with a default YOLO model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SkalskiP SkalskiP Awaiting requested review from SkalskiP SkalskiP is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /