Supervised Learning - Classification Week 9 Challenge

Neural networks, in the world of finance, assist in the development of such processes as time-series forecasting, algorithmic trading, securities classification, credit risk modeling, and constructing proprietary indicators and price derivatives. A neural network works similarly to the human brain’s neural network. A “neuron” in a neural network is a mathematical function that collects and classifies information according to a specific architecture. The network bears a strong resemblance to statistical methods such as curve fitting and regression analysis.

 

 

2) What is deep learning?

 

Deep learning is a type of machine learning that uses algorithms meant to function in a manner similar to the human brain. Deep learning represents the very cutting edge of artificial intelligence (AI) instead of teaching computers to process and learn from data (which is how machine learning works), with deep learning the computer trains itself to process and learn from data.

 

This is all possible thanks to layers of ANNS, Remember that ANN in its simplest form has only three layers, Well an  ANN that is made up of more than three layers i.e an input layer, an output layer, and multiple hidden layers is called a 'deep neural network, and this is what underpins deep learning.

 

A deep learning system is self-teaching, learning as it goes by filtering information through multiple hidden layers in a similar way to humans.

As you can see the two are closely connected in that one relies on the other to function without neural networks there would be no deep learning. Many recent advances in AI were made possible by deep learning. From recommendations on streaming services to voice assistant technologies to autonomous driving, the ability to identify patterns and classify many different types of information is crucial for processing vast amounts of data with little to no human input.

While the original goal for AI was broadly to make machines able to do things that would otherwise require human intelligence, the idea has been refined in the decades since. François Chollet, the AI researcher at Google and creator of the machine learning software library Keras, says: “Intelligence is not a skill in itself, it’s not about what you can do, but how well and how efficiently you can learn new things."1

Deep learning is focused on improving the process of having machines learn new things. With rule-based AI and ML, a data scientist determines the rules and data set features to include in models, which drives how those models operate. With deep learning, the data scientist feeds raw data into an algorithm. The system then analyses that data, without specific rules or features preprogrammed into it. Once the system makes its predictions, they are checked against a separate set of data for accuracy. The level of accuracy of these predictions – or lack thereof – then informs the next set of predictions the system makes.“Deep” refers to the many layers the neural network accumulates over time, with performance improving as the network gets deeper. Each level of the network processes its input data in a specific way, which then informs the next layer. So the output from one layer becomes the input for the next.

 

3) What is the role of the activation function?

The activation function decides whether a neuron should be activated or not by calculating the weighted sum and further adding bias to it. The purpose of the activation function is to introduce non-linearity into the output of a neuron. We know, the neural network has neurons that work in correspondence with weight, bias, and their respective activation function. In a neural network, we would update the weights and biases of the neurons on the basis of the error at the output. This process is known as back-propagation. Activation functions make the back-propagation possible since the gradients are supplied along with the error to update the weights and biases. A neural network without an activation function is essentially just a linear regression model. The activation function does the non-linear transformation to the input making it capable to learn and perform more complex tasks.

The activation function calculates a weighted total and then adds bias to it to decide whether a neuron should be activated or not. The Activation Function’s goal is to introduce non-linearity into a neuron’s output.

A Neural Network without an activation function is basically a linear regression model in Deep Learning, since these functions perform non-linear computations on the input of a Neural Network, enabling it to learn and do more complex tasks. Therefore, studying the derivatives and application of activation functions, also as analyzing the pros and drawbacks of each activation function, is essential for selecting the proper type of activation function that may give non-linearity and accuracy in a particular Neural Network model.

Non-linear activation functions: Without an activation function, a Neural Network is just a linear regression model. The activation function transforms the input in a non-linear way, allowing it to learn and as well as accomplish more complex tasks.Simply put, an activation function is a function that is added into an artificial neural network in order to help the network learn complex patterns in the data. When comparing with a neuron-based model that is in our brains, the activation function is at the end deciding what is to be fired to the next neuron. That is exactly what an activation function does in an ANN as well. It takes in the output signal from the previous cell and converts it into some form that can be taken as input to the next cell

 

 

4) What are the different types of activation functions?

1) Linear Function

2) The sigmoid function

3) Tanh Function

4) softmax function

5) Binary step function

6) Relu function

7) Leaky Relu function

8) Parameterised Relu

9) Exponential Linear Unit

10) Swish function

 

5) What do you understand by backpropagation?

Backpropagation, or backward propagation of errors, is an algorithm that is designed to test for errors working back from output nodes to input nodes. It is an important mathematical tool for improving the accuracy of predictions in data mining and machine learning. Essentially, backpropagation is an algorithm used to calculate derivatives quickly.

Artificial neural networks use backpropagation as a learning algorithm to compute a gradient descent with respect to weight values for the various inputs. By comparing desired outputs to achieved system outputs, the systems are tuned by adjusting connection weights to narrow the difference between the two as much as possible. Backpropagation algorithms are used extensively to train feedforward neural networks in areas such as deep learning. They efficiently compute the gradient of the loss function with respect to the network weights. This approach eliminates the inefficient process of directly computing the gradient with respect to each individual weight. It enables the use of gradient methods, like gradient descent or stochastic gradient descent, to train multilayer networks and update weights to minimize loss.

The difficulty of understanding exactly how changing weights and biases affect the overall behavior of an artificial neural network was one factor that held back more comprehensive use of neural network applications, arguably until the early 2000s when computers provided the necessary insight.

Today, backpropagation algorithms have practical applications in many areas of artificial intelligence (AI), including OCR, natural language processing, and image processing.

Backpropagation is one of the important concepts of a neural network. Our task is to classify our data best. For this, we have to update the weights of parameters and bias, but how can we do that in a deep neural network? In the linear regression model, we use gradient descent to optimize the parameter. Similarly here we also use a gradient descent algorithm using Backpropagation.

For a single training example, the Backpropagation algorithm calculates the gradient of the error function. Backpropagation can be written as a function of the neural network. Backpropagation algorithms are a set of methods used to efficiently train artificial neural networks following a gradient descent approach that exploits the chain rule.

The main features of Backpropagation are the iterative, recursive, and efficient method through which it calculates the updated weight to improve the network until it is not able to perform the task for which it is being trained. Derivatives of the activation function to be known at network design time are required for Backpropagation.