Combining Algorithm Unrolling with Self-Supervised Learning: Image Super-Resolution

Jørgensen, Mads Arnløv ; Larsen, Andreas Kühne ; Jónhardsson, Magnus

4. semester

Mathematical Engineering, Master

2023

2023-06-02

86 pages

Deep learning methods have shown to outperform model-based methods at image upscaling, and exhibit state-of-the-art performance. However, deep learning techniques suffer from drawbacks such as immense data requirements, computational costs, lack of interpretability/explainability, and overfitting. In an attempt to address these issues, the compatibility of algorithm unrolling and self-supervised learning is explored. First, the consequences of utilising the SSL framework data2vec to train a network inspired by ISTA-Net is examined. Then, replacing the linear projection in the encoder of vision transformers with LISTA and pre-training using the masked autoencoder framework, is investigated. Results for the first approach indicate that pre-training an ISTA-Net network using data2vec, might lead to increased generality in scarcely annotated scenarios, however strict attributions remain impossible. The results for the second approach indicate an increased performance in all the experiments. However, to solidify this result, an ablation study on how much can be attributed to the unrolled algorithm over an increased parameterisation, must be conducted.

AI ; self-supervised learning ; algorithm unrolling ; super-resolution ; compressive sensing ; math ; engineering ; machine learning ; signal processing

Download
View record in AAU Student Projects