Deep learning is all about neural networks. The structure of a neural network is like any other kind of network; there is an interconnected web of nodes, which are called neurons, and the edges that join them together.
A neural network’s main function is to receive a set of inputs, perform progressively complex calculations, and then use the output to solve a problem.
Neural nets are used for classification tasks where an object can fall into one of at least two different categories. Unlike other networks like a social network, a neural network is highly structured and comes in layers. As you have seen in the above image, The first layer is the input layer, the final layer is the output layer, and all layers in between are referred to as hidden layers. The circles in the image are known as nodes and the arrow lines that are connecting these nodes are known as synapses or weights. A neural network is like a function; it takes a few inputs and calculates an output by using bias values.
Neural networks are used for lots of different applications, For example, say you needed to predict if a patient is sick or healthy, and all you have are their height, weight, age, and body temperature. The classifier would receive this data about the patient from input layer, process it, and fire out a confidence score from output layer. Depending on the training, a high score would mean a high confidence that the patient is sick, and a low score would suggest that they are healthy or wise verse.
So A set of inputs is passed to the first hidden layer, the activations from that layer are passed to the next layer and so on, until you reach the output layer, where the results of the classification are determined by the scores at each node. This happens for each set of inputs.
So before giving you more details about neural networks, I’m gonna give you a quick overview of the problem of classification.
Classification is the process of categorizing a group of objects, while only using some basic data features that describe them. There are lots of classifiers available today – like Logistic Regression, Support Vector Machines, Naive Bayes, and of course, neural networks. The firing of a classifier, or activation as its commonly called, produces a score.
The series of events starting from the input where each activation is sent to the next layer, and then the next, all the way to the output, is known as forward propagation, Feedforward neural net.
In a feedforward network, the information moves in only one direction – forward – from the input nodes, through the hidden nodes (if any) and to the output nodes. There are no cycles or loops in the network  (this property of feed forward networks is different from Recurrent Neural Networks in which the connections between the nodes form a cycle).
The first neural nets were born out of the need to address the inaccuracy of an early classifier, the perceptron. It was shown that by using a layered web of perceptrons, the accuracy of predictions could be improved. As a result, this new breed of neural nets was called a Multi-Layer Perceptron or MLP. Since then, the nodes inside neural nets have replaced perceptrons with more powerful classifiers, but the name MLP has stuck.
Types of Feedforward neural net:
- Single Layer Perceptron – This is the simplest feedforward neural network and does not contain any hidden layer.
- Multi Layer Perceptron – A Multi Layer Perceptron has one or more hidden layers.
Multi Layer Perceptrons are mostly use now, since they are more useful than Single Layer Perceptons for practical applications today, A Multi Layer Perceptron (MLP) contains one or more hidden layers (apart from one input and one output layer). While a single layer perceptron can only learn linear functions, a multi layer perceptron can also learn non – linear functions.
The process of improving a neural net’s accuracy is called training, just like with other machine learning methods. Here’s that forward prop again – to train the net, the output from forward prop is compared to the output that is known to be correct, and the cost is the difference of the two.
The point of training is to make that cost as small as possible, across millions of training examples. To do this, the net tweaks the weights and biases step by step until the prediction closely matches the correct output. Once trained well, a neural net has the potential to make accurate predictions each time.