Machine Learning and Its Application

Overview

Neural networks have dramatically changed the capabilities of computers to perform tasks such as image classification and processing, image generation, voice recognition, game playing, and many others. We intend to:

• go over the development of neural networks,

• study the primary architectures and training methods,

• learn some of the mathematical results associated with neural networks.

Easy read

• Machine learning: Trends, perspectives, and prospects

List of Books and Courses

• Andrew Ng's Machine learning course on Coursera

• Stanford CS231n: Convolutional Neural Network

• Stanford CS229: Machine Learning

• A Brief Introduction to Neural Networks: by David Kriesel

• Overview of many topics in neural networks: by Ian Goodfellow, et al.

Theory on Machine Learning

• Efficient Processing of Deep Neural Networks: A Tutorial and Survey

• Statistical Modeling: The Two Cultures

• The Perceptron: A Probabilistic Model

• Approximation by superpositions of a sigmoidal function

• Top 10 algorithms in data mining

Architecture

• AlexNet: ImageNet Classification with Deep Convolutional Neural Networks

• Neocognitron: Convolutional Neural Networks

• ResNet: Deep Residual Learning for Image Recognition

• U-Net: Convolutional Networks

• YOLO: You Only Look Once

• Generative Adversarial Nets (GAN)

• VGG16: Very Deep Convolutional Networks

Regularization

• Dropout: A Simple Way to Prevent Neural Networks from Overfitting

• Recurrent Neural Network Regularization

Optimizer and Learning Rate

• An overview of gradient descent optimization algorithms

• Adam: A Method for Stochastic Optimization

• Cyclical Learning Rates for Training Neural Networks

• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

• No More Pesky Learning Rates

• Revisiting Small Batch Training for Deep Neural Network

• Don't Decay The Learning Rate, Increase The Batch Size