Regression Based Multi-View Zebrafish Tracking
Author
Gudiksen, Mathias
Term
4. term
Publication year
2021
Submitted on
2021-06-03
Pages
62
Abstract
Zebrafish are increasingly used in biomedical studies because their nervous systems share many features with humans. To understand how medicines affect behavior, researchers need precise, automated descriptions of how each fish moves in a tank. This thesis adapts recent multi-object tracking techniques from pedestrian tracking to follow multiple zebrafish at once. An object detector locates each fish in every frame, and the detector’s regression head is used to link these detections into smooth two-dimensional paths in two camera views (top and front). The two sets of 2D paths are then combined to estimate each fish’s 3D position by triangulation—that is, inferring depth from two viewpoints. A key challenge is identity swaps, where the algorithm accidentally exchanges the identities of two fish when they cross or look similar. We show that frequent identity swaps severely degrade 3D triangulation and lead to poor performance on a benchmark dataset. The thesis proposes a method to detect and reduce such swaps by sharing information between the two camera views across the sequence. The results indicate strong potential for detecting identity swaps, but the current strategy for correcting the affected trajectories (“tracklets”) does not yet yield improvements. Further work is needed to adjust tracks around detected swaps so that 3D reconstruction becomes more robust.
Zebrafisk bruges i stigende grad i biomedicinske forsøg, fordi deres nervesystem har mange ligheder med menneskers. For at forstå, hvordan medicin påvirker adfærden, er der behov for en præcis og automatiseret beskrivelse af, hvordan hver fisk bevæger sig i akvariet. Denne afhandling tilpasser nye metoder til sporing af flere objekter fra fodgænger-sporing for at følge flere zebrafisk på én gang. En objektdetektor lokaliserer fiskene i hvert billede, og regressionsdelen af detektoren bruges til at kæde disse fund sammen til jævne todimensionelle spor i to kameraperspektiver (set oppefra og forfra). De to sæt 2D-spor kombineres derefter for at beregne hver fisks 3D-position ved triangulering, altså at udlede dybde ud fra to synsvinkler. En central udfordring er identitetsskift, hvor algoritmen forveksler to fisk og bytter deres identiteter, f.eks. når de krydser hinanden eller ligner hinanden. Vi viser, at hyppige identitetsskift forringer 3D-trianguleringen markant og giver dårlig ydeevne på et benchmark-datasæt. Afhandlingen foreslår en metode til at opdage og mindske sådanne skift ved at dele information mellem de to kameraperspektiver gennem sekvensen. Resultaterne viser lovende evne til at opdage identitetsskift, men den nuværende strategi til at rette de berørte spor ("tracklets") giver endnu ikke forbedringer. Der er behov for mere arbejde med at justere spor omkring de detekterede skift, så 3D-rekonstruktionen bliver mere robust.
[This apstract has been rewritten with the help of AI based on the project's original abstract]
Keywords