Novel Techniques for Robust and Generalizable Machine Learning

Lemkhenter, Abdelhak (2023). Novel Techniques for Robust and Generalizable Machine Learning (Submitted). (Dissertation)

Archive Assignment_1.zip - Other Restricted to registered users only Available under License BORIS Standard License. Download (62MB)

Neural networks have transcended their status of powerful proof-of-concept machine
learning into the realm of a highly disruptive technology that has revolutionized many
quantitative fields such as drug discovery, autonomous vehicles, and machine translation. Today, it is nearly impossible to go a single day without interacting with a neural network-powered application. From search engines to on-device photo-processing,
neural networks have become the go-to solution thanks to recent advances in computational hardware and an unprecedented scale of training data. Larger and less
curated datasets, typically obtained through web crawling, have greatly propelled the
capabilities of neural networks forward. However, this increase in scale amplifies certain challenges associated with training such models. Beyond toy or carefully curated
datasets, data in the wild is plagued with biases, imbalances, and various noisy components. Given the larger size of modern neural networks, such models run the risk of
learning spurious correlations that fail to generalize beyond their training data.
This thesis addresses the problem of training more robust and generalizable machine learning models across a wide range of learning paradigms for medical time series
and computer vision tasks. The former is a typical example of a low signal-to-noise
ratio data modality with a high degree of variability between subjects and datasets.
There, we tailor the training scheme to focus on robust patterns that generalize to new
subjects and ignore the noisier and subject-specific patterns. To achieve this, we first
introduce a physiologically inspired unsupervised training task and then extend it by
explicitly optimizing for cross-dataset generalization using meta-learning. In the context of image classification, we address the challenge of training semi-supervised models
under class imbalance by designing a novel label refinement strategy with higher local
sensitivity to minority class samples while preserving the global data distribution.
Lastly, we introduce a new Generative Adversarial Networks training loss. Such
generative models could be applied to improve the training of subsequent models in the
low data regime by augmenting the dataset using generated samples. Unfortunately,
GAN training relies on a delicate balance between its components, making it prone
mode collapse. Our contribution consists of defining a more principled GAN loss whose
gradients incentivize the generator model to seek out missing modes in its distribution.

Interest & Impact

Downloads

0 since deposited on 02 May 2024
0 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item