Matrix Definiteness Visualization using MATLAB

This will be to see the definiteness of a matrix using Quadratic form surface of a matrix.

We will be using the 2x2 matrices for our implemntation as the 'Quadrratic form surface' of a N dimention matrix is in an (N+1) dimention. Since we can only visualize in 3D we will be using a 2D matrix so the 'Quadratic form surface' is in a 3D plot.

Quadratic form of a Matrix:

• applies to only square matrix
• leads to a property of square matrices called the definiteness of a matrix

Algebraic Perspective:

`=w^TxxSxxw = "a scalar"`

dimenstions of matrices/vectors `=> D(w) = 1xxm`

`=>D(S) = mxxm`

`=>D(w^T) = mxx1`

what it represents: 'Energy' in matrix S at each coordinate over the coodinate space defined by the vector 'w'. The value can be Positiv,Negative or both which depends on the fundamental feature of the matrix and the values of the matrizx elements.

If 'S' matrix is definite: `=>(w^TSw)^T = w^TS^Tw^(T T)`

`=w^TS^Tw` `("since S is symmetric" =>S^T=S)`

`=w^TSw` `("same as the definite form we started with")`

Geometric Perspective: consider it to be a function which take a Matric and a vector as an input `=> f(S,w) = w^TSw = epsilon`

every S will have a 'unique' surface but each will have only one surface for all the different valuies that can be put as elements in w. in a 2D space the value of it can be considered as the height in the z-axis as we will plot later to see its various types.

The normalized Quadratic From

as the quadratic form has the form `=w^TxxSxxw` finding the Directions along which the 'Energy ' is Maximum and Minimum can be difficult. Since the values of the Quadratic from not only depends on the Matrix 'S' but also on the Vector 'w'

`w " for which " argmax{w^TSw}`

So for example the Maximum energy can have a solution of inflating the vector 'w' to a very large value and that is a Trivial Solution or make vector 'w' very small for trivial solution of very small values, this leads to the problem of how to compare 2 different matrices . So the sollution +Inf ,-Inf or 0 are trivial.

Normalized quadratic form tells us only about the matrix and is not influenced by the elements in the vector 'w'. This motivates for a noirmalization factor that will allow us to explore the Quadratic form independent of the vector 'w'

Normlaize the QF by magnitude of w: `argmax{(w^TSw)/(w^Tw)}`

so here 'w' will only provide the coordinate system to study the energy in the matrix S, we are not interested in the magnitude of 'w'

There will be only one specific direction in which the 'normalized quadratic form' is maximum and minimum(2D).These are along the directions defined by the Eigen Vectors of the matrix S.

Eigen vecors and The Quadratic Form surface

the eigen vectors of a symmetric matrix point along the ridges and valleys pf the normalized quadratic form.

ex: a 2D matrix will have 2 eigen values and 2 eigen vectors. So the normalized QF has 2 importantg directions (maximum and minimum energy corresponding to the bigger and smaller eigne value respectively)

remember eigen value equation: `=>Av =lambdav`

Definiteness of a Matrix

it refers oto the sign of energy landscape. Has impications on the the inverse of a square matrix.

5 categories : from the eigen perspective of the QF and the signs of the eigen values.

Ignoring origin[w = 0 ] as it will always give the trivial solution of 0.

Category	Geometry	Eigen Values	Inveritbility
Positive Definite	Always Positive	All Positive	Definitely !
Positive Semi Definite	Positive and 0	All non -negative	No !
Indefinite	Postive and Negative	Positive and Negative	Possibly
Negative Semi Definite	Negative and 0	All non Positive	No !
Negative Definite	Always Negative	All Negative	Definitely !

All Semi-Definite Matrices: Singular, the inverse does not exist for the matrix. At least one eigen value `lambda = 0`. This means that in the QF surface one direction will be along 0 energy.

Illustration in MATLAB

Creating 5 aquare matrices to depict the 5 different catgories of the definiteness of square matrices

creating range of the vecor 'w' for plotting the quadratic from surface. and also matrix to store the values of the surface of the Quadratoic form.

Running the for double for loop to plot each cordinate point in the range of 'w' from [-2 2] and another for loop to loop over each of the 5 square matrices in our structure. Plotting each Quadratic from surface for comparison.

Note: we have plotted the 'raw' or non-normalized Quadratic form here for the matrices.

for 2x2 matrices ,using the signs of the eigen values to decide the category in which a particular matrix lies in

and Giveing title to plot accordingly

plotted results of Quadratic from matrices

Getting a closer look

Positive definite matrix: always Positive (values>0)

Positive Semi definite: All non negative (values>=0)

Indefinite Matrix (Both Negative and Positive)

Negative Definite :Always negative ( values<0)

Negative Semi-definite :Always non-Positve (values<=0)