Currently, this is just a place to store notes in prep for upcoming interviews with a few different firms doing various AI applications.

Convolution Neural Networks (CNN)

Inspired by Multilayered Perceptron (MLP) and the animal visual cortex. "Designed to use minimal amounts of preprocessing. They have wide applications in image and video recognition, recommender systems and natural language processing"

Multilayered Perceptron (MLP) "a type of feedforward artificial neural network model that maps sets of input data onto a set of appropriate outputs. An MLP consists of multiple layers of nodes in a directed graph, with each layer fully connected to the next one. Except for the input nodes, each node is a neuron (or processing element) with a nonlinear activation function. MLP utilizes a supervised learning technique called backpropagation for training the network. MLP is a modification of the standard linear perceptron and can distinguish data that are not linearly separable." from Wikipedia

"A feedforward neural network is an artificial neural network wherein connections between the units do not form a cycle. As such, it is different from recurrent neural networks." wiki

"Backpropagation, an abbreviation for "backward propagation of errors", is a common method of training artificial neural networks used in conjunction with an optimization method such as gradient descent. The method calculates the gradient of a loss function with respect to all the weights in the network, so that the gradient is fed to the optimization method which in turn uses it to update the weights, in an attempt to minimize the loss function.

Backpropagation requires a known, desired output for each input value in order to calculate the loss function gradient. It is therefore usually considered to be a supervised learning method, although it is also used in some unsupervised networks such as autoencoders. It is a generalization of the delta rule to multi-layered feedforward networks, made possible by using the chain rule to iteratively compute gradients for each layer. Backpropagation requires that the activation function used by the artificial neurons (or "nodes") be differentiable." wiki

Radial Basis Function

"In the field of mathematical modeling, a radial basis function network is an artificial neural network that uses radial basis functions as activation functions. The output of the network is a linear combination of radial basis functions of the inputs and neuron parameters. Radial basis function networks have many uses, including function approximation, time series prediction, classification, and system control." wiki

In Python you could use neupy or scipy.interpolate.Rbf

Random Forests

Can use 'from sklearn import tree', AND pandas to organize data going into the trees. Graphviz can be used to visualize resulting trees.

• Decision trees are very susceptible to overfitting
  • construct many trees in a 'forest' and have them all 'vote' towards the outcome classification
    • MUST randomly sample data used to make each tree!
    • Also, randomize the attributes each tree is fitting.

Bayesian Methods: Concepts

Bayes' Theorem (not covering it here as it is in my PhD and MSc theses)

One good real-world application is in a spam filter. Naive Bayes' can be used to develop a model that can discriminate normal (Ham) emails from garbage (Spam). Lots of ways to improve it, but works fairly well in a basic sense.

Spam Classifier/Filter with Naive Bayes

Supervised learning.

Steps

• Read in emails and their classification of ham/spam the bulk of the code
• Vectorize emails to numbers representing each word
• Get a functional object that will perform Multinomial Naive Bayes' from sklearn
• Fit vectorized emails
• Check it worked with test cases

Visualizing data

Visualization of the data can enable one to discover much more than standardized statistics like mean, median or mode... an article highlighting this for the Anscombe’s Quartet problem.

• An interesting option, is a library built ontop of matplotlib seaborn
• An entire article discussing major 7 available Python based tools here