Supervised Learning - Classification Week 9 Challenge

Q1.Neural networks are a class of machine learning algorithms used to model complex patterns in datasets using multiple hidden layers and non-linear activation functions.

A neural network is a computational model that is inspired by the way biological neural networks, such as the human brain, work. It is a type of machine learning algorithm designed to recognize patterns and make decisions, by processing and learning from data.A neural network consists of interconnected artificial neurons, which are organized into layers. The most common type of neural network is a feedforward neural network.

1. Input Layer: This layer receives the raw data or features that the network will process. Each neuron in the input layer represents a feature of the input data.

2. Hidden Layers: These layers come between the input and output layers and contain artificial neurons (also called nodes or units) that perform complex calculations and transformations of the input data. Hidden layers are where most of the "learning" occurs, as they learn to recognize patterns and features in the data.

3. Output Layer: The output layer provides the final result or prediction generated by the neural network. The number of neurons in the output layer depends on the type of problem the network is designed to solve. For example, in a binary classification task, there might be one neuron for each class (e.g., "yes" or "no"). In a regression task, there might be a single neuron producing a numerical value.

Q2. 

Deep learning is a subfield of machine learning and artificial intelligence that focuses on the development and application of artificial neural networks, particularly deep neural networks. Deep learning techniques are characterized by the use of neural networks with multiple layers (hence the term "deep"), also known as deep neural networks.Deep learning is a type of artificial intelligence (AI) that teaches computers to process data in a way that is inspired by the human brain. Deep learning models can recognize complex patterns in pictures, text, sounds, and other data to produce accurate insights and predictions.

1. Deep Neural Networks
2. Feature Learning
3. Representation Learning
4. Scalability
5. Neural Network Architectures
6. Training with Backpropagation

Q3.The role of an activation function is to add non-linearity to a neural network. It helps the network learn complex patterns in the data.

Activation function is a function that is added into an artificial neural network in order to help the network learn complex patterns in the data.

Q4.

1. Step Function:

   • Output: 0 for inputs less than a threshold, 1 for inputs greater than or equal to the threshold.
   • Use: Historically used in early neural network models but rarely used in modern networks due to its discontinuous nature.

2. Sigmoid Function (Logistic Function):

   • Output: S-shaped curve, values between 0 and 1.
   • Use: Often used in the output layer for binary classification problems, where the output can be interpreted as a probability.

3. Hyperbolic Tangent (tanh) Function:

   • Output: S-shaped curve, values between -1 and 1.
   • Use: Similar to the sigmoid, but centered at 0, making it useful in hidden layers of neural networks.

4. Rectified Linear Unit (ReLU):

   • Output: Linear for positive values, 0 for negative values.
   • Use: One of the most popular activation functions, known for its simplicity and effectiveness, often used in hidden layers of deep neural networks.

5. Leaky ReLU:

   • Output: Linear for positive values, a small, non-zero slope for negative values.
   • Use: Addresses the "dying ReLU" problem where ReLU units can get stuck during training.

6. Parametric ReLU (PReLU):

   • Output: Similar to Leaky ReLU but with the slope for the negative part learned during training.
   • Use: Can help improve training for deep networks.

7. Exponential Linear Unit (ELU):

   • Output: An exponential curve for negative values and linear for positive values.
   • Use: Provides a smoother transition than ReLU and can help with training stability.

8. Scaled Exponential Linear Unit (SELU):

   • Output: A self-normalizing function that tends to keep the mean and variance of activations close to 0 and 1, respectively.
   • Use: Particularly useful for deep networks and can potentially improve training stability.

9. Swish:

   • Output: A smooth, non-monotonic function.
   • Use: Designed to combine the best characteristics of ReLU and sigmoid-like functions, potentially leading to better performance.

10. Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM):

• Output: Specialized activation functions used in recurrent neural networks (RNNs) to model sequential data.
• Use: Specifically designed to capture long-range dependencies in sequences and are commonly used in tasks like natural language processing.

Q5.

 Backpropagation, short for "backward propagation of errors," is a supervised learning algorithm used to train artificial neural networks, including deep neural networks. Its primary purpose is to adjust the network's internal parameters, such as weights and biases, so that the network can learn to make accurate predictions or classifications based on input data. Backpropagation is a critical component of the training process for neural networks.

1. Forward Pass
2. Loss Calculation
3. Backward Pass (Backpropagation)
4. Weight and Bias Updates
5. Iteration