Supervised Learning - Classification Week 9 Challenge

1. What is a Neural Network?

Ø A Neural Network is a computational model inspired by the way biological neural networks in the human brain work.

Ø It consists of interconnected nodes or artificial neurons organized into layers. These layers include an input layer, one or more hidden layers, and an output layer.

Ø Neural networks are trained on data to learn patterns and relationships, making them capable of performing tasks such as classification, regression, and pattern recognition.

2. What is deep learning?

Ø Deep Learning is a subset of machine learning that involves neural networks with multiple layers, also known as deep neural networks.

Ø The term "deep" refers to the depth of the network, which has more than one hidden layer.

Ø Deep learning excels at automatically learning hierarchical features from data, making it suitable for complex tasks such as image recognition, natural language processing, and speech recognition.

3. What is the role of the activation function?

Ø The activation function in a neural network introduces non-linearity to the model, enabling it to learn complex relationships and patterns in the data.

Ø Without activation functions, the neural network would reduce to a linear model, unable to capture the non-linearities present in many real-world datasets.

Ø Activation functions determine the output of each neuron based on its input and define the range of values that the neuron can produce.

4. What are the different types of activation functions?

There are several types of activation functions used in neural networks, each with its characteristics. Common activation functions include:

Ø Sigmoid Function (Logistic): S-shaped curve, output range between 0 and 1.

Ø Hyperbolic Tangent (tanh): Similar to the sigmoid but with a range between -1 and 1.

Ø Rectified Linear Unit (ReLU): Outputs the input for positive values, zero for negative values.

Ø Leaky ReLU: Similar to ReLU but allows a small, non-zero gradient for negative inputs.

Ø Parametric ReLU (PReLU): Generalization of Leaky ReLU with a learnable parameter.

Ø Exponential Linear Unit (ELU): Similar to ReLU but smooth for negative values.

Ø Softmax: Used in the output layer for multi-class classification, converts raw scores into probabilities.

5. What do you understand by backpropagation?

Backpropagation, short for "backward propagation of errors," is a supervised learning algorithm used to train neural networks. It involves updating the weights of the network in the opposite direction of the gradient of the loss function with respect to the weights. The process consists of two main steps:

Ø Forward Pass: Input data is fed through the network to generate predictions. Each layer's output is calculated using the weights and activation functions.

Ø Backward Pass (Backpropagation): The error between the predicted output and the true target is calculated. The gradient of the error with respect to the weights is computed layer by layer using the chain rule of calculus.

Ø The weights are then updated to minimize the error, and this process is repeated iteratively during training.