|
|
This project develops an end-to-end system for 2D metrics extraction from ski jumper skeletons (+ skis), starting from raw competition footage from the Ski-TB Dataset.
The main objectives are:
- Create one of the biggest annotated ski jumping dataset: manually label a 23-keypoint skeleton (body + skis) on competition videos.
- Extract biomechanical metrics: compute 2D geometric and dynamic metrics (joint angles, V-style, body-ski inclination, flight stability, landing quality, …) and correlate them with judges' scores and athletes distance.
- Train the Ski Pose Estimation model: use a transformer-based deep learning model to predict ski positions given only the body skeleton.
Fan Engagement Enhancement: Real-time jump analysis for broadcasts, providing viewers with instant visual feedback on jump technique and performance metrics.
Coaching Tool: Detailed biomechanical feedback for athletes and coaching staff, enabling frame-by-frame technique comparison and improvement tracking.
We annotated 32 ski jumps from the Ski-TB Dataset using a custom 23-keypoint skeleton model (15 body joints + 8 ski keypoints). All annotations were created with Roboflow and the complete dataset comprises 9798 frames.
Each jump video contains approximately 350 frames. Out of these, around 60 key frames are manually annotated in Roboflow in COCO format. The remaining frames are then completed through the following automated pipeline:
- Extract: parse jump-specific annotations from the exported COCO file.
- Interpolate: linearly interpolate keypoints between annotated frames to obtain annotations for all ~350 frames.
- Normalize: normalize keypoint coordinates relative to the bounding box, making them resolution and position independent for downstream metric computation.
- Visualize, generate annotated images and overlay videos for quality inspection.
annotation_preprocessing/
├── main.py # Workflow orchestrator
├── raw_annotations/ # Input COCO annotations from Roboflow
└── utils/ # Function classes used for annotation pipeline
The annotation pipeline outputs data into the dataset/ folder:
dataset/
├── frames/JP00XX/ # Raw video frames organized by jump ID
├── annotations/JP00XX/ # Processed COCO annotations + visualization overlays
├── keypoints_dataset.csv # Normalized keypoints for all frames, ready for metrics computation
├── jump_phases_SkiTB.csv # Frame ranges for each jump phase (take-off, v-style, flight, landing, telemark)
└── JP_data.csv # Athlete metadata: name, nationality, scores, judges' evaluations, hills info
Starting from the annotated and normalized keypoints, we computed a set of 2D biomechanical metrics to quantitatively describe each jump. These metrics account for the inherent limitations of a 2D perspective (e.g., foreshortening, camera angle variability) by favoring dynamic measures (velocities, standard deviations) over static absolute angles where possible.
| Category | Metrics | Description |
|---|---|---|
| V-Style | avg_v_style_front, avg_v_style_back |
Ski opening angle from front and back views |
| Body-Ski Angle | avg_body_ski_angle |
Inclination between body axis and ski axis during flight |
| Take-off | takeoff_knee_angle, knee_peak_velocity |
Knee extension angle and explosive velocity at jump |
| Flight Stability | flight_std, flight_jitter |
Variability and micro-instability of body-ski angle |
| Landing | landing_hip_velocity, landing_knee_compression, telemark_scissor |
Landing softness and telemark quality |
 |
 |
 |
 |
 |
| Take Off knee angle | Body Ski Angle | V style Angle | Symmetry Index | Telemark Scissor Ratio |
metrics/
├── core_metrics/ # Per-frame geometric metrics (angles, positions)
│ ├── metrics_computation.py # Main metrics computation script
│ ├── metrics_per_frame.csv # Metrics for each frame
│ ├── metrics_summary_per_jump.csv # Aggregated metrics per jump
│ └── timeseries_metrics/ # Time-series dynamics (velocity, jitter)
├── correlations/ # Statistical correlation with judges' scores
│ ├── correlation_analysis.py
│ └── *.csv / *.png # Results and heatmaps
├── data_quality/ # Outlier detection and data validation
├── metrics_visualizations/ # Overlay visualizations on frames
├── profile_analysis/ # Top vs. flop athlete comparisons
└── style_penalty_model/ # Ridge regression model predicting style penalties
We are aware that some of the results obtained from the metrics and correlation analyses are not all statistically significant, for two main reasons:
- Small dataset: with only 32 annotated jumps, the sample size limits the statistical power of any analysis.
- Low performance heterogeneity: All jumps in the dataset are winners or podiums in FIS World Cup competitions, representing the absolute elite standard of the sport. Since these are top-level athletes, the data exhibits low variance, making difficult to capture distinct patterns or significant differences between jumps, as the biomechanical margins between competitors are minimal. A more heterogeneous dataset would likely yield more significant and differentiated results.
The SkiPoseModel is a transformer-based deep learning model originally introduced in this paper. Its goal is to predict the position of the 8 ski keypoints given only the body skeleton of the jumper (with ski joints masked during training).
We adapted the model to our custom 23-keypoint dataset and trained it on a total of 9798 samples (6858 train, 1470 validation, 1470 test).
The model predicts the 8 ski keypoints (4 per ski). After inference, a PCA-based linearization step is applied to force the predicted ski points onto a straight line, producing more physically plausible results. Here are some visualizations of the results.
 |
 |
 |
 |
SkiPoseModel/
├── main.py # Training / testing / demo entry point
├── model.py # AdaptationNetwork (PyTorch Lightning module)
├── datamodule.py # SkijumpDataModule and SkijumpDataset
├── transformer.py # Transformer architecture blocks
├── preprocess.py # COCO JSON → pickle preprocessing
├── postprocess_visualize.py # Visualization & ski linearization
├── domainadapt_flags.py # Configuration flags
├── requirements.txt # Model-specific dependencies
├── dataset/ # Raw dataset (COCO JSON)
├── dataset_preprocessed/ # Preprocessed splits (train.pkl, val.pkl, test.pkl)
└── results/ # Predictions, checkpoints, visualizations
├── checkpoints/ # Saved model weights
├── test_results.pkl # Raw test predictions
└── test_results_linearized.pkl # Post-processed (linearized skis)
- Python: 3.9+
- GPU (required for SkiPoseModel): NVIDIA GPU with CUDA 11.8+
# 1. Clone the repository
git clone https://github.com/elisa-negrini/SkiProject-SportTech.git
cd SkiProject-SportTech
# 2. Create virtual environment
python -m venv sport_tech_env
# Windows
sport_tech_env\Scripts\activate
# Linux/Mac
source sport_tech_env/bin/activate
# 3. Install dependencies
pip install -r requirements.txt
⚠️ Note: The dataset and model checkpoints are not included in the repository due to size constraints.
Downdload the content of this Google Drive
After downloading, place files as follows:
├── annotation_preprocessing/
│ └── raw_annotations/ ← Extract raw_annotations here
├── dataset/
│ ├── frames/ ← Extract frames here
│ └── annotations/ ← Extract annotations here
│
├── SkiPoseModel/
│ ├── results/ ← Place results here
│ └── dataset_preprocessed/ ← Place dataset_preprocessed here
│
├── metrics/
└── metrics_visualizations/ ← Place frame_overlays
If you have new annotations on Roboflow (following our skeleton scheme), export the COCO file and replace the content of annotation_preprocessing/raw_annotations/, then run:
python annotation_preprocessing/main.pyThis will extract, filter, interpolate, normalize, and visualize the annotations. Output is saved to the dataset/ folder, as described above.
⚠️ Training and inference require an NVIDIA GPU with CUDA support.
cd SkiPoseModel
# Preprocess dataset (COCO JSON → pickle)
python preprocess.py
# Train the model
python main.py --mode train
# Test the model
python main.py --mode test --load_checkpont "enter_the_checkpoint_path"
# Post-process & visualize predictions (ski linearization)
python postprocess_visualize.pyCompute biomechanical metrics and run analyses using the scripts in the metrics/ folder:
# Core geometric metrics
python metrics/core_metrics/metrics_computation.py
# Correlation analysis with judges' scores
python metrics/correlations/correlation_analysis.py
# Data quality checks
python metrics/data_quality/data_quality_check.py
# Profile analysis (top vs. flop)
python metrics/profile_analysis/profile_analysis.py
# Style penalty model
python metrics/style_penalty_model/style_penalty_model.pyThe dashboard provides two main screens:
- Gallery Explorer: browse and filter the dataset by jump, athlete, and phase; explore annotated frames with skeleton overlays.
- Metric Analysis: visualize metrics filtered by athlete and metric type, with summary statistics.
streamlit run dashboard/Dashboard.py- Small Sample Size: The dataset is limited to 32 annotated jumps, reducing statistical power for correlation analysis.
- Low Heterogeneity: Most samples represent 1st place or podium finishes by elite athletes, resulting in low variance in performance metrics.
- 2D Perspective: Metrics are computed in 2D space, making them susceptible to foreshortening and perspective distortions.
- Uncalibrated Cameras: Footage sourced from broadcast streams has varying camera angles (multi-view) and lacks intrinsic calibration, affecting metric consistency.
- Hill Variability: Jumps were recorded on different hills with varying geometric profiles, introducing uncontrolled environmental variables.
- 3D Pose Lifting: Implement 3D lifting techniques to convert 2D keypoints into 3D space, obtaining true biomechanical angles independent of camera perspective.
- Dataset Expansion: Incorporate jumps from junior categories or training sessions to introduce performance variance and improve model generalization.
- Real-Time Application: Optimize the pipeline for real-time inference to enable live visual feedback for TV broadcasts or coaching.
| Name | ID | |
|---|---|---|
| Diego Conti | diego.conti@studenti.unitn.it | 257632 |
| Elisa Negrini | elisa.negrini@studenti.unitn.it | 258422 |
| Federico Molteni | federico.molteni@studenti.unitn.it | 243030 |
- Ski-TB Dataset: Base dataset for ski jumping videos
- Roboflow: Annotation platform
- FIS Ski Jumping Rule Book: Rule book to validate our metrics and performance analysis
- Kinematic Analysis of the Take-Off and Start of the Early Flight Phase on a Large Hill (HS-134 m) during the 2009 Nordic World Ski Championships: Validates our take-off metrics
- Optimal flight technique for V-style ski jumping: Validates our flight phase metrics
- Ground Reaction Forces and Kinematics of Ski Jump Landing Using Wearable Sensors and Landing in Ski Jumping: A Review About its Biomechanics and the Connected Injuries: Validate our landing metrics
Sport Tech 2025/2026, University of Trento