Supervised Learning - Classification Week 9 Challenge
1.What is a Neural Network?
An artificial neural network (ANN) is a system of hardware and/or software patterned after the operation of neurons in the
human brain.
Neural networks, also known as artificial neural networks (ANNs) or simulated neural networks (SNNs), are a subset of
machine learning and are at the heart of deep learning algorithms.
Artificial neural networks (ANNs) are comprised of a node layers, containing an input layer, one or more hidden layers, and
an output layer. Each node, or artificial neuron, connects to another and has an associated weight and threshold. If the
output of any individual node is above the specified threshold value, that node is activated, sending data to the next
layer of the network. Otherwise, no data is passed along to the next layer of the network.
Neural networks rely on training data to learn and improve their accuracy over time. However, once these learning algorithms
are fine-tuned for accuracy, they are powerful tools in computer science and artificial intelligence, allowing us to
classify and cluster data at a high velocity. Tasks in speech recognition or image recognition can take minutes versus hours
when compared to the manual identification by human experts. One of the most well-known neural networks is Google’s search
algorithm.
Working of neural network:
Each individual node as its own linear regression model, composed of input data, weights, a bias (or threshold), and an
output. The formula would look something like this:
∑wixi + bias = w1x1 + w2x2 + w3x3 + bias
output = f(x) = 1 if ∑w1x1 + b>= 0; 0 if ∑w1x1 + b < 0
Once an input layer is determined, weights are assigned. These weights help determine the importance of any given variable,
with larger ones contributing more significantly to the output compared to other inputs. All inputs are then multiplied by
their respective weights and then summed. Afterward, the output is passed through an activation function, which determines
the output. If that output exceeds a given threshold, it “fires” (or activates) the node, passing data to the next layer
in the network. This results in the output of one node becoming in the input of the next node. This process of passing
data from one layer to the next layer defines this neural network as a feedforward network.
Neural network could make increasingly complex decisions depending on the output of previous decisions or layers.
the goal is to minimize our cost function to ensure correctness of fit for any given observation. As the model adjusts its
weights and bias, it uses the cost function and reinforcement learning to reach the point of convergence, or the local
minimum. The process in which the algorithm adjusts its weights is through gradient descent, allowing the model to
determine the direction to take to reduce errors (or minimize the cost function). With each training example, the
parameters of the model adjust to gradually converge at the minimum.
Most deep neural networks are feedforward, meaning they flow in one direction only, from input to output. However, you can
also train your model through backpropagation; that is, move in the opposite direction from output to input.
Backpropagation allows us to calculate and attribute the error associated with each neuron, allowing us to adjust and fit
the parameters of the model(s) appropriately.
Types of Neural Network
*The perceptron is the oldest neural network, created by Frank Rosenblatt in 1958. It has a single neuron and is the
simplest form of a neural network
*Feedforward neural networks, or multi-layer perceptrons (MLPs), are what we’ve primarily been focusing on within this
article. They are comprised of an input layer, a hidden layer or layers, and an output layer. While these neural networks
are also commonly referred to as MLPs, it’s important to note that they are actually comprised of sigmoid neurons, not
perceptrons, as most real-world problems are nonlinear. Data usually is fed into these models to train them, and they are
the foundation for computer vision, natural language processing, and other neural networks.
*Convolutional neural networks (CNNs) are similar to feedforward networks, but they’re usually utilized for image
recognition, pattern recognition, and/or computer vision. These networks harness principles from linear algebra,
particularly matrix multiplication, to identify patterns within an image.
*Recurrent neural networks (RNNs) are identified by their feedback loops. These learning algorithms are primarily
leveraged when using time-series data to make predictions about future outcomes, such as stock market predictions or sales
forecasting.
2. What is deep learning?
Deep learning is a subset of machine learning, which is essentially a neural network with three or more layers. These
neural networks attempt to simulate the behavior of the human brain—albeit far from matching its ability—allowing it to
“learn” from large amounts of data. While a neural network with a single layer can still make approximate predictions,
additional hidden layers can help to optimize and refine for accuracy.
Deep learning drives many artificial intelligence (AI) applications and services that improve automation, performing
analytical and physical tasks without human intervention. Deep learning technology lies behind everyday products and
services (such as digital assistants, voice-enabled TV remotes, and credit card fraud detection) as well as emerging
technologies (such as self-driving cars).
Deep learning neural networks, or artificial neural networks, attempts to mimic the human brain through a combination of
data inputs, weights, and bias. These elements work together to accurately recognize, classify, and describe objects
within the data.
Deep neural networks consist of multiple layers of interconnected nodes, each building upon the previous layer to refine
and optimize the prediction or categorization. This progression of computations through the network is called forward
propagation. The input and output layers of a deep neural network are called visible layers. The input layer is where the
deep learning model ingests the data for processing, and the output layer is where the final prediction or classification
is made.
Another process called backpropagation uses algorithms, like gradient descent, to calculate errors in predictions and then
adjusts the weights and biases of the function by moving backwards through the layers in an effort to train the model.
Together, forward propagation and backpropagation allow a neural network to make predictions and correct for any errors
accordingly. Over time, the algorithm becomes gradually more accurate.
deep learning algorithms are incredibly complex, and there are different types of neural networks to address specific
problems or datasets.
3.What is the role of the activation function?
Whats a activation function:
It’s just a thing function that you use to get the output of node. It is also known as Transfer Function.
Why we use Activation functions with Neural Networks?
It is used to determine the output of neural network like yes or no. It maps the resulting values in between 0 to 1 or -1
to 1 etc. (depending upon the function).
The Activation Functions can be basically divided into 2 types:
*Linear Activation Functions
*Non-linear Activation Functions
To put in simple terms, an artificial neuron calculates the ‘weighted sum’ of its inputs and adds a bias.
Now the value of net input can be any anything from -inf to +inf. The neuron doesn’t really know how to bound to value and
thus is not able to decide the firing pattern. Thus the activation function is an important part of an artificial neural
network. They basically decide whether a neuron should be activated or not. Thus it bounds the value of the net input.
4.What are the different types of activation functions?
Several different types of activation functions are used in Deep Learning. Some of them are explained below:
*Step function : Step Function is one of the simplest kind of activation functions. In this, we consider a threshold value
and if the value of net input say y is greater than the threshold then the neuron is activated.
*Sigmoid function : Sigmoid function is a widely used activation function.
This is a smooth function and is continuously differentiable. The biggest advantage that it has over step and linear
function is that it is non-linear. This is an incredibly cool feature of the sigmoid function. This essentially means
that when I have multiple neurons having sigmoid function as their activation function – the output is non linear as
well. The function ranges from 0-1 having an S shape.
*ReLU function : The ReLU function is the Rectified linear unit. It is the most widely used activation function.
The main advantage of using the ReLU function over other activation functions is that it does not activate all the
neurons at the same time. What does this mean ? If you look at the ReLU function if the input is negative it will
convert it to zero and the neuron does not get activated.
*Leaky ReLU : Leaky ReLU function is nothing but an improved version of the ReLU function.Instead of defining the Relu
function as 0 for x less than 0, we define it as a small linear component of x.
5.
Backpropagation exist for other artificial neural networks (ANNs), and for functions generally. These classes of algorithms
are all referred to generically as "backpropagation".[2] In fitting a neural network, backpropagation computes the gradient
of the loss function with respect to the weights of the network for a single input–output example, and does so efficiently,
unlike a naive direct computation of the gradient with respect to each weight individually. This efficiency makes it
feasible to use gradient methods for training multilayer networks, updating weights to minimize loss.
Backpropagation computes the gradient in weight space of a feedforward neural network, with respect to a loss function.