Adaline (Adaptive Linear Neural) :

An artificial neural network inspired by the human neural system is a network used to process the data which consist of three types of layer i.e input layer, the hidden layer, and the output layer. The basic neural network contains only two layers which are the input and output layers. The layers are connected with the weighted path which is used to find net input data. In this section, we will discuss two basic types of neural networks Adaline which doesn’t have any hidden layer, and Madaline which has one hidden layer.

1. Adaline (Adaptive Linear Neural) :

A network with a single linear unit is called Adaline (Adaptive Linear Neural). A unit with a linear activation function is called a linear unit. In Adaline, there is only one output unit and output values are bipolar (+1,-1). Weights between the input unit and output unit are adjustable. It uses the delta rule i.e
$w_{i}(new)=w_{i}(old) + α(t-y_{in})x_{i}$
, where
$w_{i}$

$y_{in}$
and
$t$
are the weight, predicted output, and true value respectively.
The learning rule is found to minimize the mean square error between activation and target values. Adaline consists of trainable weights, it compares actual output with calculated output, and based on error training algorithm is applied.

Workflow:

Adaline

First, calculate the net input to your Adaline network then apply the activation function to its output then compare it with the original output if both the equal, then give the output else send an error back to the network and update the weight according to the error which is calculated by the delta learning rule. i.e $w_{i}(new)=w_{i}(old) + α(t-y_{in})x_{i}$ , where $w_{i}$ $y_{in}$ and $t$ are the weight, predicted output, and true value respectively.

Architecture:

Adaline

In Adaline, all the input neuron is directly connected to the output neuron with the weighted connected path. There is a bias b of activation function 1 is present.

Algorithm:

Step 1: Initialize weight not zero but small random values are used. Set learning rate α.

Step 2: While the stopping condition is False do steps 3 to 7.

Step 3: for each training set perform steps 4 to 6.

Step 4: Set activation of input unit xi = si for (i=1 to n).

Step 5: compute net input to output unit

$y_{in} = \sum w_ix_i + b$

Here, b is the bias and n is the total number of neurons.

Step 6: Update the weights and bias for i=1 to n

$w_{i}(new) = w_{i}(old) + \alpha(t-y_{in})x_{i} \\ b(new)=b(old) + α(t-y_{in})$

and calculate

$error : (t-y_{in})^2$

when the predicted output and the true value are the same then the weight will not change.

Step 7: Test the stopping condition. The stopping condition may be when the weight changes at a low rate or no change.

Implementations

Problem: Design OR gate using Adaline Network?

Solution :

Initially, all weights are assumed to be small random values, say 0.1, and set learning rule to 0.1.
Also, set the least squared error to 2.
The weights will be updated until the total error is greater than the least squared error.

Calculate the net input
$y_{in} = \sum w_i x_i + b$

$y_{in} = 0.1\times1 + 0.1\times1 +0.1 =0.3$ (when x1=x2=1)

Now compute, (t-yin)=(1-0.3)=0.7
Now, update the weights and bias

$w_i(new) = w_i(old) + α(t-y_{in})x_i \\ w_1(new) = 0.1 + 0.1(1-0.3)1 = 0.17 \\ w_2(new) = 0.1 + 0.1(1-0.3)1 = 0.17$

$b(new)=b(old) + α(t-y_{in}) \\ b(new)= 0.1 + 0.1(1-0.3) = 0.17$

calculate the error
$error = (t-y_{in})^2 = 0.7^2 = 0.49$

Similarly, repeat the same steps for other input vectors and you will get.

This is epoch 1 where the total error is 0.49 + 0.69 + 0.83 + 1.01 = 3.02 so more epochs will run until the total error becomes less than equal to the least squared error i.e 2.

Adaline (Adaptive Linear Neural) :

1. Adaline (Adaptive Linear Neural) :

Workflow:

Architecture:

Algorithm:

Implementations

Post a Comment

What is NFA and DFA?

Comparison between Deterministic Finite Automaton (DFA) and Non-deterministic

Contact Form