Supervised Learning - Classification Week 9 Challenge

1. Neural Network:

Ans: A neural network is a type of machine-learning model that is inspired by the structure and function of the human brain. It is composed of layers of interconnected "neurons," which process and transmit information. Neural networks are commonly used for tasks such as image recognition, natural language processing, and decision-making. They can learn to recognize patterns in data and make predictions or decisions without being explicitly programmed.

2. What is deep learning?

Ans: Deep learning is a subset of machine learning that is concerned with the development of neural networks with multiple layers, known as deep neural networks. These networks are able to learn increasingly abstract representations of the input data through a process called hierarchical feature extraction. This allows them to learn and improve on tasks such as image and speech recognition, natural language processing, and decision-making. Deep Learning algorithms are used to train these neural networks with large datasets, allowing the network to learn and improve on its own without human intervention.

3. What is the role of the activation function?

Ans:

The activation function in a neural network serves to introduce non-linearity into the output of a neuron. Without an activation function, a neural network would consist of only linear operations and would be less expressive and less powerful.

Activation functions are applied element-wise to the output of a neuron, resulting in a new output that is then used as input for the next layer. Activation functions are designed to introduce non-linearity into the output of a neuron, allowing the neural network to learn and represent a much more complex and varied range of patterns and relationships in the data.

There are several types of activation functions that can be used, such as the sigmoid, ReLU (Rectified Linear Unit), and tanh (hyperbolic tangent) functions. Each activation function has its own properties and is suitable for different types of tasks and architectures.

4. What are the different types of activation functions?

Ans:

• Sigmoid: This function maps any input value to a value between 0 and 1. It is useful for output layers in which the output must be a probability.

• ReLU (Rectified Linear Unit): This function maps any negative input value to 0 and any positive input value to itself. It is widely used in deep networks because of its computational efficiency and improved performance.

• tanh (hyperbolic tangent): This function maps any input value to a value between -1 and 1. It is similar to the sigmoid function, but maps to a wider range of values.

• softmax: This function is used in output layers of multi-class classification problems, it maps the output of last hidden layer to probability values across multiple classes.

• LeakyReLU: It is an extension to ReLU, it allows small negative values to pass through, it is useful in situations where ReLU neurons may die.

• ELU (Exponential Linear Unit): This function is similar to ReLU but it has negative values for negative inputs.

• Swish: It is a new activation function developed in 2017, it's defined as (x * sigmoid(x)) , it has been shown to be more effective than ReLU in some cases.

5. What do you understand by backpropagation?

Ans:

Backpropagation is the process by which neural networks learn from the input data. It is the primary algorithm used to train neural networks and is a supervised learning method.

It works by first making a prediction based on the input data, and then comparing that prediction to the true output. The difference between the prediction and the true output is called the error, and the goal of backpropagation is to minimize this error by adjusting the weights and biases of the neural network.

The process of backpropagation is divided into two phases: the forward phase and the backward phase. In the forward phase, the input data is passed through the neural network, and the output is calculated. In the backward phase, the error is calculated and then "propagated" back through the network, adjusting the weights and biases in such a way as to minimize the error.

This process is repeated many times, with the network adjusting its weights and biases each time based on the error, until the error is minimized to a satisfactory level or the model is converged. This is an iterative process and is also known as Gradient Descent.

Backpropagation is a powerful and widely used algorithm that allows neural networks to learn from data and improve their performance over time.