Python Version TensorFlow OpenCV MoveNet Performance License

A real-time exercise form analysis system using Google's MoveNet pose estimation model to evaluate movement quality and count repetitions for fitness exercises.

This project implements an intelligent exercise monitoring system that uses computer vision to analyze human movement patterns. The system provides real-time feedback on exercise form, counts repetitions, and scores movement quality for common exercises like squats and pull-ups.

• Utilizes Google's MoveNet model to detect 17 human keypoints with high accuracy
• Compares detected poses against reference movements using cosine similarity matching
• Currently supports squats and pull-ups with extensible architecture for additional exercises
• Automatically counts completed exercise repetitions based on movement patterns
• Provides real-time scoring based on movement accuracy and form
• Works with webcam input for real-time analysis
• Supports batch processing of pre-recorded exercise videos
• Saves annotated videos with pose overlays and performance metrics

graph TD
 A[Video Input] --> B[Frame Capture]
 B --> C[MoveNet Model]
 C --> D[Pose Detection]
 D --> E[Keypoint Extraction]
 E --> F[Pose Normalization]
 F --> G[Similarity Matching]
 G --> H[Exercise Classification]
 H --> I[Repetition Counting]
 I --> J[Quality Scoring]
 J --> K[Real-time Display]
 K --> L[Video Output]
 
 M[Reference Database] --> G
 N[Exercise Models] --> H
 O[Scoring Algorithms] --> J

1. Input Processing: Captures video frames from camera or file input
2. Pose Detection: Applies MoveNet model to extract 17 skeletal keypoints
3. Normalization: Normalizes pose vectors for scale and position invariance
4. Similarity Matching: Compares poses against reference database using cosine similarity
5. State Tracking: Monitors exercise phases (up/down positions) for repetition counting
6. Scoring: Calculates form quality scores based on anatomical angle analysis

• MoveNet Integration: TensorFlow Hub model wrapper for pose estimation
• Similarity Engine: Ball Tree implementation for efficient pose matching
• Exercise Evaluators: Specialized scoring algorithms for different exercise types
• Video Pipeline: Multi-threaded video processing with real-time display

• Python 3.8+
• CUDA-compatible GPU (recommended for optimal performance)
• Webcam (for real-time analysis)

pip install tensorflow
pip install tensorflow-hub
pip install opencv-python
pip install scikit-learn
pip install numpy
pip install matplotlib

1. Clone the repository:

   git clone https://github.com/Magicherry/Pose_Estimation.git
   cd Pose_Estimation

2. Download MoveNet models (automatically handled by TensorFlow Hub)

3. Prepare reference pose data in base_data/ directory following the existing structure

python main.py

Modify the following parameters in main.py:

• use_camera: Set to True for live camera input, False for video file processing
• sport_type: Choose between "squat" or "pull_up"
• model_name: Select "movenet_lightning" (faster) or "movenet_thunder" (more accurate)

• 1: Switch to squat analysis mode
• 2: Switch to pull-up analysis mode
• ESC: Exit the application

Test your camera setup:

python cameraTest.py

Pose_Estimation/
├── main.py # Main application entry point
├── cameraTest.py # Camera functionality testing
├── setup.py # Build configuration
├── base_data/ # Reference pose datasets
│ ├── squat/
│ │ ├── up/ # Reference images for squat up position
│ │ └── down/ # Reference images for squat down position
│ └── pull_up/
│ ├── up/ # Reference images for pull-up up position
│ └── down/ # Reference images for pull-up down position
├── movenet/ # MoveNet model files and utilities
│ ├── movenet.py # MoveNet model wrapper
│ └── singlepose/ # Pre-trained model weights
├── utils/
│ └── utils.py # Utility functions for pose visualization
└── result/ # Output directory for processed videos

The system normalizes detected poses to handle variations in:

• Scale: Different distances from camera
• Position: Various locations within frame
• Orientation: Minor rotational differences

The squat evaluation measures movement depth by analyzing the relationship between hip, knee, and ankle positions during the exercise.

Core Algorithm:

• Calculate vertical distances from ankle to hip and ankle to knee
• Compute depth ratio: knee_ankle_distance / hip_ankle_distance
• Convert to 0-100 score scale

Scoring Criteria:

The pull-up evaluation tracks face position relative to arm joints to determine completion quality.

Core Algorithm:

• Calculate face center from 5 facial keypoints (nose, eyes, ears)
• Compare face position to elbow and wrist levels
• Score based on range of motion completion

Scoring Zones:

# Squat scoring example
def calculate_squat_score(hip_y, knee_y, ankle_y):
 hip_ankle_dist = abs(hip_y - ankle_y)
 knee_ankle_dist = abs(knee_y - ankle_y)
 depth_ratio = min(knee_ankle_dist / (hip_ankle_dist + 1e-6), 1.0)
 return int(depth_ratio * 100)
# Pull-up scoring example 
def calculate_pullup_score(face_y, elbow_y, wrist_y):
 if face_y < wrist_y:
 return 100
 elif face_y >= elbow_y:
 return 0
 else:
 return int((elbow_y - face_y) / (elbow_y - wrist_y) * 100)

stateDiagram-v2
 [*] --> Idle
 Idle --> DownDetected: Pose matches "down"
 DownDetected --> DownConfirmed: Sustained > 10 frames
 DownConfirmed --> UpDetected: Pose matches "up"
 UpDetected --> UpConfirmed: Sustained > 10 frames
 UpConfirmed --> RepComplete: Count++
 RepComplete --> Idle: Reset for next rep
 
 DownDetected --> Idle: Noise/false detection
 UpDetected --> DownConfirmed: Return to down

Optimization Strategies:

• Model Selection: Choose between Lightning (speed) and Thunder (accuracy) variants
• Multi-threading: Separate threads for video input and processing
• Efficient Matching: Ball Tree data structure for fast similarity searches
• Memory Management: Optimized frame processing pipeline

1. Create reference pose data in base_data/new_exercise/
2. Implement exercise-specific scoring function
3. Add exercise type to main configuration
4. Update similarity matching parameters if needed

Implement new scoring functions following the pattern:

def update_custom_exercise_score(pose, best_score):
 # Analyze pose keypoints
 # Calculate exercise-specific metrics 
 # Return updated best score
 return new_score

The system provides real-time visual feedback with the following elements:

• Main Video Display: Live camera feed with pose skeleton overlay (white lines, colored keypoints)
• Performance Metrics: Exercise type, repetition count, current score, and processing FPS
• Status Information: Frame counter, pose classification (up/down), and system status

The system logs important events with timestamps:

[12:34:56.789] INFO: MoveNet model loaded successfully
[12:34:57.123] RESULT: Squat #8 completed, score: 87/100
[12:34:58.456] WARNING: Processing FPS dropped to 18

This project is licensed under the Mozilla Public License 2.0 - see the LICENSE file for details.

• Google Research for the MoveNet pose estimation model
• TensorFlow team for the model hub infrastructure
• OpenCV community for computer vision tools