Supervised Learning - Classification Week 9 Challenge

1)What is a Neural Network?

Ans 1)

A neural network is a network or circuit of biological neurons, or, in a modern sense, an artificial neural networks, composed of artificial neurons or nodes.Thus, a neural network is either a biological neural networks, made up of biological neurons, or an artificial neural network, used for solving artificial intelligence (AI) problems. The connections of the biological neuron are modeled in artificial neural networks as weights between nodes. A positive weight reflects an excitatory connection, while negative values mean inhibitory connections. All inputs are modified by a weight and summed. This activity is referred to as a linear combination. Finally, an activation function controls the amplitude of the output. For example, an acceptable range of output is usually between 0 and 1, or it could be −1 and 1.

These artificial networks may be used for predictive modelling, adaptive control and applications where they can be trained via a dataset. Self-learning resulting from experience can occur within networks, which can derive conclusions from a complex and seemingly unrelated set of information.

2)What is deep learning?

Ans2)

Deep learning is a subset of machine learning which is essentially a neural network with three or more layers.These neural
network attempt to simulate the behavior of the human brain-albeit far from matching its ability allowing it to learn
from large amounts of data.

3)What is the role of the activation function?

Ans3)an activation function is a function that is added into an artificial neural network in order to help the network learn complex patterns in the data. When comparing with a neuron-based model that is in our brains, the activation function is at the end deciding what is to be fired to the next neuron.

4)What are the different types of activation functions?

Ans 4)

Different activation function used with neural networks are as follows:

1. Sigmoid Function

2. Linear Activation Function

3. Rectified Linear Unit Function

4. Hyperbolic Tangent Function

1). Linear Function :-

• Equation : Linear function has the equation similar to as of a straight line i.e. y = x
• No matter how many layers we have, if all are linear in nature, the final activation function of last layer is nothing but just a linear function of the input of first layer.
• Range : -inf to +inf
• Uses : Linear activation function is used at just one place i.e. output layer.
• Issues : If we will differentiate linear function to bring non-linearity, result will no more depend on input “x” and function will become constant, it won’t introduce any ground-breaking behavior to our algorithm.
• 2). Sigmoid Function :-
  • It is a function which is plotted as ‘S’ shaped graph.
  • Equation : A = 1/(1 + e^-x)
  • Nature : Non-linear. Notice that X values lies between -2 to 2, Y values are very steep. This means, small changes in x would also bring about large changes in the value of Y.
  • Value Range : 0 to 1
  • Uses : Usually used in output layer of a binary classification, where result is either 0 or 1, as value for sigmoid function lies between 0 and 1 only so, result can be predicted easily to be 1 if value is greater than 0.5 and 0 otherwise.
• Tanh Function :- The activation that works almost always better than sigmoid function is Tanh function also knows as Tangent Hyperbolic function. It’s actually mathematically shifted version of the sigmoid function. Both are similar and can be derived from each other.
• Softmax Function :- The softmax function is also a type of sigmoid function but is handy when we are trying to handle mult- class classification problems.

• ReLU Function

  ReLU stands for Rectified Linear Unit.

  Although it gives an impression of a linear function, ReLU has a derivative function and allows for backpropagation while simultaneously making it computationally efficient.

  The main catch here is that the ReLU function does not activate all the neurons at the same time.

  The neurons will only be deactivated if the output of the linear transformation is less than 0.

5. What do you understand by backpropagation?

ANS: While training a neural network at first some random weights are chosen as initial values and the loss function is computed with these weights. Obviously, the loss is high and the network has a poor performance. This loss is fed backwards to fine-tune the neural network's weights further over a number of iterations, so that the loss reduces. This method of passing on the information backwards to improve the algorithm is called Backpropagation.