Supervised Learning - Classification Week 9 Challenge

1.

A neural network is a computational model inspired by the structure and functioning of the human brain. It consists of interconnected nodes, called neurons, organized in layers. Each neuron receives input signals, processes them, and produces an output signal which can be passed to other neurons. Neural networks are trained on data to perform tasks such as classification, regression, and pattern recognition.

2.

Deep learning is a subfield of machine learning that focuses on using neural networks with many layers (hence the term "deep") to learn from data. Deep learning algorithms have shown remarkable success in various domains, including image recognition, natural language processing, and game playing.

3.

The activation function plays a crucial role in a neural network by introducing non-linearity into the model. Without activation functions, the neural network would behave like a linear regression model, making it unable to learn complex patterns in the data. Activation functions decide whether a neuron should be activated or not, based on the weighted sum of its inputs. This activation determines the output of the neuron, which is then passed to the next layer of the neural network.

4.

There are several types of activation functions used in neural networks, including:

1. Sigmoid : Maps the input to a range between 0 and 1, suitable for binary classification tasks.

2. ReLU (Rectified Linear Unit) : Sets all negative values to zero, leaving positive values unchanged. It's widely used in deep learning due to its simplicity and efficiency.

3. Tanh (Hyperbolic Tangent) : Similar to the sigmoid function but maps the input to a range between -1 and 1, offering stronger gradients during training.

4. Leaky ReLU : Similar to ReLU but allows a small, positive gradient for negative inputs, preventing the dying ReLU problem.

5. Softmax : Used in the output layer for multi-class classification tasks, it converts raw scores into probabilities.

5.

Backpropagation is an algorithm used to train neural networks by adjusting the weights of the connections between neurons. It works by calculating the gradient of the loss function with respect to the weights of the network using the chain rule of calculus. This gradient is then used to update the weights in such a way that the loss function is minimized, effectively optimizing the network's performance. Backpropagation is an iterative process that involves passing the input forward through the network, calculating the error, and then propagating this error backward to update the weights. Through repeated iterations of this process on a training dataset, the neural network learns to make better predictions.