Supervised Learning - Classification Week 9 Challenge

1. What is a Neural Network?

A Neural Network (NN) is a computational model inspired by the human brain, designed to recognize patterns and make decisions. It consists of:

• Input Layer: Takes in raw data.
• Hidden Layers: Processes data using weights and biases.
• Output Layer: Produces the final prediction.

Neural networks are used in applications like image recognition, speech processing, and natural language understanding.

---

2. What is Deep Learning?

Deep Learning is a subset of Machine Learning that uses multi-layered neural networks (deep neural networks) to automatically extract patterns from large amounts of data. It enables:

• Complex pattern recognition
• Minimal feature engineering
• Better generalization on large datasets

Deep learning powers applications like autonomous driving, facial recognition, and language translation.

---

3. What is the Role of the Activation Function?

The activation function in a neural network introduces non-linearity, allowing the model to learn complex relationships in the data. Without an activation function, the model would behave like a simple linear regression.

Roles of an activation function:

• Introduces non-linearity
• Controls neuron activation
• Enables deep learning models to learn complex features

---

4. What are the Different Types of Activation Functions?

Activation functions determine the output of a neuron based on weighted inputs. Common types include:

Activation Function	Formula	Use Case
Sigmoid	$f(x) = \frac{1}{1+e^{-x}}$​	Binary classification
Tanh (Hyperbolic Tangent)	$f(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}$​	Better than sigmoid, ranges from -1 to 1
ReLU (Rectified Linear Unit)	$f(x) = max(0, x)$	Most commonly used in deep networks
Leaky ReLU	$f(x) = max(0.01x, x)$	Handles zero gradients in ReLU
Softmax	$f(x_i) = \frac{e^{x_i}}{\sum e^{x_j}}$​​	Multi-class classification

---

5. What do you Understand by Backpropagation?

Backpropagation (Backprop) is a key algorithm in training neural networks. It adjusts weights and biases by computing the gradient of the loss function with respect to each parameter.

Steps in backpropagation:

1. Forward Pass: Compute predicted outputs.
2. Compute Loss: Calculate the difference between predicted and actual values.
3. Backward Pass: Compute gradients using the chain rule.
4. Update Weights: Adjust weights using gradient descent.

Backpropagation helps the model learn by minimizing the error iteratively. It is widely used in deep learning frameworks like TensorFlow and PyTorch.