Week 4 - Genetic Algorithm

AIM: To optimize the stalagamite function and find global maxima using Genetic Algorithm.

OBJECTIVE:

1) write a program for stalagamite function.

2) write a program to optimize the stalagmite function and find global maxima using genetic algorithm.

3) Explain genetic algorithm and its code to understand its statistical behaviour to ensure that program gives the same output, even if it is made to run several times.

THEORY:

Genetic Algorithm (GA) is a search-based optimization technique based on the principles of Genetics and Natural Selection. It is frequently used to find optimal or near-optimal solutions to difficult problems which otherwise would take a lifetime to solve. It is frequently used to solve optimization problems, in research, and in machine learning.

Optimization is the process of making something better. In any process, we have a set of inputs and a set of outputs as shown in the following figure.

Optimization refers to finding the values of inputs in such a way that we get the “best” output values. The definition of “best” varies from problem to problem, but in mathematical terms, it refers to maximizing or minimizing one or more objective functions, by varying the input parameters.

The set of all possible solutions or values which the inputs can take make up the search space. In this search space, lies a point or a set of points which gives the optimal solution. The aim of optimization is to find that point or set of points in the search space.

Nature has always been a great source of inspiration to all mankind. Genetic Algorithms (GAs) are search based algorithms based on the concepts of natural selection and genetics. GAs are a subset of a much larger branch of computation known as Evolutionary Computation.

GAs were developed by John Holland and his students and colleagues at the University of Michigan, most notably David E. Goldberg and has since been tried on various optimization problems with a high degree of success.

In GAs, we have a pool or a population of possible solutions to the given problem. These solutions then undergo recombination and mutation (like in natural genetics), producing new children, and the process is repeated over various generations. Each individual (or candidate solution) is assigned a fitness value (based on its objective function value) and the fitter individuals are given a higher chance to mate and yield more “fitter” individuals. This is in line with the Darwinian Theory of “Survival of the Fittest”.

In this way we keep “evolving” better individuals or solutions over generations, till we reach a stopping criterion.

Genetic Algorithms are sufficiently randomized in nature, but they perform much better than random local search (in which we just try various random solutions, keeping track of the best so far), as they exploit historical information as well.

Advantages of GAs

GAs have various advantages which have made them immensely popular. These include −

• Does not require any derivative information (which may not be available for many real-world problems).

• Is faster and more efficient as compared to the traditional methods.

• Has very good parallel capabilities.

• Optimizes both continuous and discrete functions and also multi-objective problems.

• Provides a list of “good” solutions and not just a single solution.

• Always gets an answer to the problem, which gets better over the time.

• Useful when the search space is very large and there are a large number of parameters involved.

Limitations of GAs

Like any technique, GAs also suffer from a few limitations. These include −

• GAs are not suited for all problems, especially problems which are simple and for which derivative information is available.

• Fitness value is calculated repeatedly which might be computationally expensive for some problems.

• Being stochastic, there are no guarantees on the optimality or the quality of the solution.

• If not implemented properly, the GA may not converge to the optimal solution.

Basic Terminology

Before beginning a discussion on Genetic Algorithms, it is essential to be familiar with some basic terminology which will be used throughout this tutorial.

• Population − It is a subset of all the possible (encoded) solutions to the given problem. The population for a GA is analogous to the population for human beings except that instead of human beings, we have Candidate Solutions representing human beings.

• Chromosomes − A chromosome is one such solution to the given problem.

• Gene − A gene is one element position of a chromosome.

• Allele − It is the value a gene takes for a particular chromosome.

Basic Structure

The basic structure of a GA is as follows −

We start with an initial population (which may be generated at random or seeded by other heuristics), select parents from this population for mating. Apply crossover and mutation operators on the parents to generate new off-springs. And finally these off-springs replace the existing individuals in the population and the process repeats. In this way genetic algorithms actually try to mimic the human evolution to some extent.

Syntax of ga:-

x = ga(problem) finds the minimum for problem

[x,fval] = ga(___), for any previous input arguments, also returns fval, the value of the fitness function at x.

[x,fval,exitflag,output] = ga(___) also returns exitflag, an integer identifying the reason the algorithm terminated, and output, a structure that contains output from each generation and other information about the performance of the algorithm

[x,fval,exitflag,output,population,scores] = ga(___) also returns a matrix population, whose rows are the final population, and a vector scores, the scores of the final population

x = ga(fun,nvars,A,b,[],[],lb,ub,nonlcon,IntCon) or x = ga(fun,nvars,A,b,[],[],lb,ub,nonlcon,IntCon,options) requires that the variables listed in IntCon take integer values.

GOVERNING EQUATION:

where,

X & Y = input variables

F(x,y) = stalagmite function

- Program for stalagmite function.

function f = stalagmite_func(input_vector)
x=input_vector(1);
y=input_vector(2);

f1x=(sin(5.1*pi*x + 0.5))^6;
f1y=(sin(5.1*pi*x +0.5))^6;

f2x=exp(-4*log(2).*((x-0.667).^2/0.64));
f2y=exp(-4*log(2).*((y-0.667).^2/0.64));


f = -1*(f1x*f1y*f2x*f2y); % multiply by -1 give maxima
end

- Program to optimize the stalagmite funtion and to find global maxima.

% Write a program to find global maxima with unbound inputs

close all
clear all
clc

% field of interest for optimization
x = linspace(0,0.6,150);
y = linspace(0,0.6,150);

% creating a 2-D matrix vector
[xx yy] = meshgrid(x,y);
num_cases = 50;


for i= 1:length(xx)
    for j=1:length(yy)
        input_vector(1)=xx(i,j);
        input_vector(2)=yy(i,j);
        f(i,j)= stalagmite_func(input_vector);
    end
end

    
% creating input data
tic % study time 
for i=1:num_cases %sample population
    [inputs,fopt(i)] = ga(@stalagmite_func,2);
    xopt(i) = inputs(1);
    yopt(i) = inputs(2);
end
toc


% plotting results
figure(1)
subplot(2,1,1)
hold on
surfc(xx,yy,f)
shading interp
plot3(xopt,yopt,-fopt,'marker','o','markersize',5,'markerfacecolor','r')
title('unbounded inputs')
xlabel('X-axis') %labelling axis
ylabel('Y-axis')
zlabel('F(x,y)')

subplot(2,1,2)
plot(-fopt)
xlabel('Iterations')
ylabel('Function maximum')

Here we find out that at each iteration the program gives different output, which is not correct so we now proceed further into genetic algorithm statistics and improve in by following two cases:-

1) As we know that genetic algorithm work on random sampling as it does not follows the input array domain therefore in order to get unique output we need to bound the function inputs.

2) In order to minimizes the range of function output we maximizes the sample population.

- Program to optimize the stalagmite funtion and to find global maxima with bounded input with default population of 50.

close all
clear all
clc

%creating 1-D inputs
x = linspace(0,0.6,150);
y = linspace(0,0.6,150);

%creating 2-D matrix
[xx yy] = meshgrid(x,y);

%sample population
num_cases = 50;


for i= 1:length(xx)
    for j=1:length(yy)
        input_vector(1)=xx(i,j);
        input_vector(2)=yy(i,j);
        f(i,j)= stalagmite_func(input_vector);
    end
end

%creating input data 
    tic
    for i=1:num_cases
        [inputs,fopt(i)] = ga(@stalagmite_func,2,[],[],[],[],[0;0],[0.6;0.6]);
        xopt(i)=inputs(1);
        yopt(i)=inputs(2);
    end
    toc
    
    %plotting results
    figure(2)
    subplot(2,1,1)
    hold on
    surfc(x,y,-f)
    shading interp
    plot3(xopt,yopt,-fopt,'marker','o','markersize',5,'markerfacecolor','r')
    title('Bounded inputs')
    xlabel('X-axis') %labelling axis
    ylabel('Y-axis')
    zlabel('F(x,y)')

    
    subplot(2,1,2)
    plot(-fopt)
    xlabel('Iteration')
    ylabel('function maximum')

- Program to optimize the stalagmite funtion and to find global maxima with bounded input and increases population to 200.

close all
clear all
clc

%creating 1-D inputs
x = linspace(0,0.6,150);
y = linspace(0,0.6,150);

%creating 2-D matrix
[xx yy] = meshgrid(x,y);

%sample population
num_cases = 50;


for i= 1:length(xx)
    for j=1:length(yy)
        input_vector(1)=xx(i,j);
        input_vector(2)=yy(i,j);
        f(i,j)= stalagmite_func(input_vector);
    end
end

%creating input data 
    tic
    for i=1:num_cases
        [inputs,fopt(i)] = ga(@stalagmite_func,2,[],[],[],[],[0;0],[0.6;0.6]);
        xopt(i)=inputs(1);
        yopt(i)=inputs(2);
    end
    toc
    
    %plotting results
    figure(2)
    subplot(2,1,1)
    hold on
    surfc(x,y,-f)
    shading interp
    plot3(xopt,yopt,-fopt,'marker','o','markersize',5,'markerfacecolor','r')
    title('Bounded inputs')
    xlabel('X-axis') %labelling axis
    ylabel('Y-axis')
    zlabel('F(x,y)')

    
    subplot(2,1,2)
    plot(-fopt)
    xlabel('Iteration')
    ylabel('function maximum')

RESULT:

With the help of bounded inputs and increased sample population from default 50 to 200, we minimizes the range of our function(f) to [0813531 0813534]

REFRENCES:

1) https://www.tutorialspoint.com/genetic_algorithms/genetic_algorithms_introduction.htm

2) https://in.mathworks.com/help/gads/ga.html

3) Prof.C.Balaji, NPTEL(iit-Madras)