We organize a talk about deep networks by Alex Lamb, who is a PhD student at the University of Montreal with Yoshua Bengio and Aaron Courville. Anyone is welcome to join the seminar.
|Title||Manifold Mixup: Encouraging Meaningful On-Manifold Interpolation as a Regularizer|
|Date||15:00-16:00, July 31 (Tue), 2018|
|Venue||1901 Meeting Room, National Institute of Informatics|
|Participation||Anyone is welome to join. Free of charge. Registration is not required, but if you are from the outside of NII, you can make a contact with kitamoto @ nii.ac.jp about how to enter the building. Access to NII.|
|Presentation||Manifold Mixup (Google Slides)|
|Abstract||Deep networks often perform well on the data manifold on which they are trained, yet give incorrect (and often very confident) answers when evaluated on points from off of the training distribution. This is exemplified by the adversarial examples phenomenon but can also be seen in terms of model generalization and domain shift. We propose Manifold Mixup which encourages the network to produce more reasonable and less confident predictions at points with combinations of attributes not seen in the training set. This is accomplished by training on convex combinations of the hidden state representations of data samples. Using this method, we demonstrate improved semi-supervised learning, learning with limited labeled data, and robustness to adversarial examples. Manifold Mixup requires no (significant) additional computation. Analytical experiments on both real data and synthetic data directly support our hypothesis for why the Manifold Mixup method improves results.|
|Bio||Alex Lamb is a PhD student at the University of Montreal with Yoshua Bengio and Aaron Courville. He is interested in making deep learning more robust, general, and useful in realistic learning scenarios. Previously he worked on the demand forecasting group at Amazon and did his undergraduate at the Johns Hopkins University. He also creates machine learning tutorials and guides on his youtube channel: https://www.youtube.com/c/TheNuttyNetterAlexLamb|