Academic Project Page

Abstract

This paper presents TeamTrack, a new benchmark dataset and algorithm for multi-object tracking (MOT) in team sports, specifically in full-pitch videos. MOT in team sports is a challenging task due to object occlusions, similar appearances, and complex movements. Existing methods often struggle to accurately track objects in these scenarios. To address this challenge, the proposed TeamTrack dataset captures diverse object appearances and movements in soccer, basketball, and handball games using high-resolution fisheye and drone cameras. The dataset includes over 4 million annotated bounding boxes and provides a comprehensive resource for developing and evaluating MOT algorithms. The paper also introduces a new MOT approach that incorporates trajectory forecasting using a graph neural network (GNN) to model complex group movement patterns. The experiments demonstrate the effectiveness of the proposed algorithm on the TeamTrack dataset.

Dataset

We present TeamTrack, a novel MOT dataset comprising over 150 minutes of high-resolution video from multiple team sports. The main characteristics of TeamTrack are the following:

Large-scale: TeamTrack introduces over 4 million annotated bounding boxes across various tracklets, making it one of the largest datasets of its kind.
Multiple Sports: TeamTrack includes matches from three team sports; soccer, basketball, and handball.
Similar Appearance and Dynamic Movement: The dataset features targets with similar appearance, dynamic movements, and frequent occlusions, offering a robust challenge for tracking algorithms.
Full Pitch Multi Angle View: Videos are recorded from two angles, top view via drones and side view with fisheye lenses, both covering the entire playing field.

Beyond its significant size and the inherent challenges it presents, TeamTrack’s unique value lies in its multi-view and full-pitch capture settings. These features enable experimentation with multi-view tracking and the use of prior information (e.g., player count, pitch dimensions) not feasible with traditional broadcast videos. Additionally, we hope the adversarial-cooperative nature of team sports can also be studied to further refine tracking techniques.

**Table 1:** Comparison of MOT datasets and TeamTrack.
Dataset	Frames	BBoxes	Domain
MOT16	11,235	292,733	Pedestrians
MOT20	13,410	2,102,385	Pedestrians
KITTI-T	10,870	65,213	Autonomous Driving
DanceTrack	105,855	-	Dance
SSET	12,000	12,000	Soccer
SN-Tracking	225,375	3,645,661	Soccer
SportsMOT	150,379	1,629,490	Soccer Basketball Volleyball
SoccerTrack	82,800	2,484,000	Soccer
TeamTrack (ours)	279,900	4,374,900	Soccer Basketball Handball

**Table 2:** Camera and video details of the multi-object tracking dataset for various sports.
Sport	Camera Type	Location	Device	Resolution	Minutes	Bounding Box Count
Soccer	Fisheye	A University, Outdoor	Z CAM E2-F8	8K	30	1,242,000
Soccer	Drone	A University, Outdoor	DJI Mavic 3	4K	30	1,242,000
Basketball	Fisheye	B University, Indoor	Z CAM E2-F6	6K	17.5	346,500
Basketball	Drone (Side View)	C University, Outdoor	DJI Mavic 3	4K	24	475,200
Basketball	Drone (Top View)	C University, Outdoor	DJI Mavic 3	4K	24	475,200
Handball	Fisheye	B University, Indoor	Z CAM E2-F6	6K	30	594,000
Total					155.5	4,374,900

Tracking Benchmarks

Below we present a direct evaluation of our proposed TeamTrack dataset by applying two state-of-the-art tracking algorithms, ByteTrack and BoT-SORT. Additionally, we compare these results with earlier benchmarks performed on the MOT17 and DanceTrack datasets to highlight the challenges posed by our TeamTrack dataset. We use the implementations from Ultralytics for both ByteTrack and BoT-SORT.

Performance of the various MOT models.

Table 3: Soccer SideView

Method	HOTA	DetA	AssA	MOTA	IDF1
BoT-SORT	58.4	62.8	54.5	84.2	73.8
ByteTrack	59.3	64.4	54.7	86.4	74.2

Table 4: Soccer TopView

Method	HOTA	DetA	AssA	MOTA	IDF1
BoT-SORT	51.9	51.1	53.3	42.7	65.7
ByteTrack	53.7	51.4	56.5	43.3	69.2

Table 5: Basketball SideView

Method	HOTA	DetA	AssA	MOTA	IDF1
BoT-SORT	75.2	79.2	71.4	94.3	85.9
ByteTrack	76.2	75.5	76.9	89.3	88.6

Table 6: Basketball SideView2

Methods	HOTA	DetA	AssA	MOTA	IDF1
BoT-SORT	47.3	67.6	33.1	80.2	50.8
ByteTrack	42.9	54.7	33.7	65.0	53.6

Table 7: Basketball TopView

Method	HOTA	DetA	AssA	MOTA	IDF1
BoT-SORT	66.3	62.7	70.3	89.0	93.9
ByteTrack	65.7	65.1	66.4	89.6	92.0

Table 8: Handball SideView

Methods	HOTA	DetA	AssA	MOTA	IDF1
BoT-SORT	75.1	75.5	74.7	91.6	89.7
ByteTrack	73.5	73.8	73.2	89.4	87.6

Performance of the various MOT models on different datasets.

Table 9: Comparison of TeamTrack with other datasets.

Dataset	HOTA	DetA	AssA	MOTA	IDF1
TeamTrack Ave.	61.9	64.2	60.2	77.2	77.5
Basketball-SV	76.2	75.5	76.9	89.3	88.6
Basketball-SV2	42.9	54.7	33.7	65	53.6
MOT17	63.1	64.5	62	80.3	77.3
DanceTrack	47.1	70.5	31.5	88.2	51.9

Change Log

2024-04-22: Released the dataset and updated arXiv links.

BibTeX

To cite this work in your academic research, please use the following BibTeX entry:

                
@article{teamtrack2023,
  title={TeamTrack: An Algorithm and Benchmark Dataset for Multi-Sport Multi-Object Tracking in Full-pitch Videos},
  author={Scott, Atom and Uchida, Ikuma and Ding, Ning and Umemoto, Rikuhei and Bunker, Rory and Kobayashi, Ren and Koyama, Takeshi and Onishi, Masaki and Kameda, Yoshinari and Fujii, Keisuke},
  journal={arXiv preprint arXiv:submit/5550700},
  year={2023}
}

TeamTrack: An Algorithm and Benchmark Dataset for Multi-Sport Multi-Object Tracking in Full-pitch Videos

Abstract

Dataset

Video Preview

Tracking Benchmarks

Change Log

BibTeX