Skip to content

Latest commit

 

History

History
20 lines (12 loc) · 1.41 KB

README.md

File metadata and controls

20 lines (12 loc) · 1.41 KB

How does training deep neural networks on a biased dataset affect the loss landscape of the network?

This repository contained the code for our project for the course "Lab Development and Application of Data Min-ing and Learning Systems: Machine Learningand Data Mining".

We investigate the loss landscape of a deep neural network trained on a biased dataset by visualizing the loss landscape using "filter-normalized random directions" as described by Li, Hao et al. “Visualizing the Loss Landscape of Neural Nets.” NeurIPS (2018).

Methodology

We trained "ResNet-20" and "ShaResNet-20" several times on "CIFAR-10 Dataset" with different types of bias (mislabeling, gaussian noise and skewness).

Implementations

  • We use a slightly modified version of keras example for ResNet and CIFAR-10
  • We implemented the visualization method in Tensorflow, its original implementation in PyTorch is available here
  • We modified the impleemntation of the ResNet to implement the ShaResNet
  • Finally we implemented the logic for adding different biases to the training dataset

Report

For detailed methodology and final results please see our final report.