A Vision Transformer Framework for Irregularly Sampled Multivariate Time Series

Roy, Nipa

3. term

Software, Master

2025

This thesis investigates a new approach to analyzing and classifying irregularly sampled multivariate time series by converting them into images and treating the task as a visual learning problem. In many real-world settings, such as healthcare monitoring, environmental sensing, and mobile data collection, measurements are recorded at uneven time intervals, vary in length, and contain substantial gaps, which makes it difficult to apply conventional models that assume regular sampling and complete observations. Standard solutions often rely on heavy preprocessing, including interpolation and imputation, which can introduce artificial structure and suppress informative irregularities. Instead, this work proposes to visualize each time series instance as a structured image composed of line plots, where each variable is drawn directly against time so that irregular sampling and sparsity are preserved as visual features. These line-graph images are arranged in grid layouts to expose both local dynamics within individual variables and broader relationships across variables. The resulting images are then analyzed using vision transformer architectures, which are well suited to capturing local detail and global context through patch-based self-attention. By relying on visual structure rather than explicit temporal alignment, the proposed framework can naturally accommodate missing observations, uneven sampling rates, and variable-length sequences without resorting to complex continuous-time modeling or expensive numerical solvers. The thesis situates this approach within a broad landscape of related work, including classical statistical time series models, neural methods for irregular data, Neural ODE-based continuous-time architectures, transformer-based numerical sequence models, graph and set-based formulations, and existing imaging techniques such as recurrence plots and spectrograms. Experimental results across multiple datasets (only briefly indicated in the excerpt) suggest that the visual modeling strategy with vision transformers is effective and robust, especially in challenging conditions with high sparsity or incomplete sensor data, and enables reuse of mature computer vision techniques for time series classification. The main contribution is a framework for generating line-graph images from irregular multivariate time series and applying vision transformers for classification, offering a simple and flexible alternative or complement to traditional time series models under irregular and incomplete data conditions.

Denne afhandling undersøger en ny tilgang til analyse og klassifikation af uregelmæssigt samplede multivariate tidsserier ved at omdanne dem til billeder og behandle dem som et visuelt læringsproblem. I mange praktiske anvendelser, eksempelvis sundhedsdata, miljømålinger og mobile sensorer, registreres målinger med ujævne tidsintervaller, varierende længder og manglende observationer, hvilket gør det vanskeligt at bruge traditionelle modeller, der forudsætter regelmæssig sampling og fuldstændige data. Konventionelle løsninger bygger ofte på omfattende forbehandling såsom interpolation og imputering, som kan tilføre kunstig struktur og fjerne vigtige uregelmæssigheder. I dette arbejde foreslås i stedet at visualisere hver tidsserie som et struktureret billede bestående af linjegrafer, hvor hver variabel tegnes direkte mod tiden, så uregelmæssig sampling og sparsitet bevares som visuelle kendetegn. Disse line-graf billeder organiseres i et grid for at fremhæve både lokale dynamikker inden for hver variabel og relationer på tværs af variabler. Billederne analyseres derefter med vision transformer-arkitekturer, som er velegnede til at udnytte både lokal detalje og global struktur gennem patch-baseret selv-opmærksomhed. Ved at basere sig på visuel struktur frem for eksplicit tidsjustering kan metoden håndtere manglende data, variable samplingrater og sekvenser med forskellig længde uden kompleks kontinuer-tidsmodellering eller tunge numeriske løsere. Afhandlingen placerer den foreslåede tilgang i forhold til eksisterende arbejde inden for klassiske statistiske tidsseriemodeller, neurale netværk til uregelmæssige data, Neural ODE-modeller, transformer-baserede sekvensarkitekturer, graf- og set-baserede tilgange samt eksisterende billedbaserede repræsentationer som recurrensplots og spektrogrammer. Eksperimentelle resultater på flere datasæt (som kun kort nævnes i uddraget) indikerer, at den visuelle ramme med vision transformers er både effektiv og robust, særligt i udfordrende scenarier med høj sparsitet eller ufuldstændige sensordata, og muliggør genbrug af modne computer vision-teknikker til tidsserieklassifikation. Afhandlingens hovedbidrag er en ramme for generering af line-graf billeder fra uregelmæssige multivariate tidsserier og anvendelsen af vision transformers til klassifikation, som tilbyder et enkelt og fleksibelt alternativ eller supplement til traditionelle tidsseriemodeller under uregelmæssige og mangelfulde dataforhold.

[This abstract has been generated with the help of AI directly from the project full text]

Download PDF
View record in AAU Student Projects

Roy, Nipa:

• A Hybrid Graph and Vision Transformer Framework for Irregularly Sampled Multivariate Time Series Classification (2026)