Neural Network (Artificial) ANN is a high-performance computing device whose core theme is inspired by biological neural networks. The human brain comprises billions of neurons, each of which is linked to several other neurons to form a network, allowing it to recognize and process images. Each biological neuron can process a variety of inputs and generate output. Neurons in the human brain are capable of making extremely complex decisions, which means they can perform several tasks parallel. All of these concepts led to the development of a computer model of the brain using an artificial neural network. The primary goal of an artificial neural network is to create a system that can perform a variety of computational tasks faster than conventional systems. Pattern recognition and classification, approximation, optimization, and data clustering are some of these functions. ANN collects a large number of units that are linked in some way to enable contact between them. These modules, also known as nodes or neurons, are basic processors that work in a parallel fashion.
Input Layer - Input features are provided to this layer. It includes information from the outside world to the network; no computation is done at this layer. Nodes here only pass on the data (features) to the hidden layer.
Hidden Layer - This layer's nodes aren't visible to the outside world; they're part of the abstraction that every neural network provides. The hidden layer computes all of the features entered via the input layer and sends the results to the output layer.
Output Layer - This layer communicates the network's acquired knowledge to the outside world.
Artificial neurons are the basic unit of a neural network. The artificial neuron takes one or more inputs and adds them together to create an output. Perceptrons are another name for artificial neurons. An artificial neuron is:
Y= Σ (weights * input) + bias
wights= It controls the signal between two neurons (or the intensity of the connection) To put it another way, a weight determines how much of an impact the input has on the output.
Bias= Constant biases are an extra input into the next layer that often has the value of one. The bias unit ensures that even though all of the inputs are zeros, the neuron will still be activated.
The activation function calculates a weighted number and then adds bias to it to determine if a neuron should be activated or not. For non-linear complex functional mappings between the inputs and the required variable, activation functions are used. The activation function's goal is to introduce non-linearity into a neuron's output.
Some commonly used activation functions are:
f(x) = 1 / 1 + exp(-x)
As per looking at the graph its range can be defined from 0 to 1.
Disadvantages:
Slow convergence Vanishing gradient problem The Sigmoid's output is not zero-centered, causing its gradient to shift in different directions.
The hyperbolic tangent function is represented as
f(x) = 1 — exp(-2x) / 1 + exp(-2x)
As per looking at the graph its range can be defined from -1 to 1.
Unlike the sigmoid function, the output of tanh function is zero-centered. But the vanishing gradient problem still prevails.
Rectified linear units function is the most commonly used function as it solves the problem that the above two functions could not solve. If the function receives any negative input, it returns 0; however, if the function receives any positive value x, it returns that value. It can be represented as
f(x)= max(0,x)
As per looking at the graph its range can be defined from 0 to infinite.
For more you can also go through this video https://www.youtube.com/watch?v=aircAruvnKk