Curve Fitting Techniques

Curve fitting is the process of constructing a curve, or mathematical functions, which possess the closest proximity to the real series of data. By curve fitting, we can mathematically construct the functional relationship between the observed dataset and parameter values etc. It is highly effective in mathematical modelling of some natural processes.

Assume some data points (x1,y1), (x2,y2), (x3,y3),...(xn,yn) and assume the best fit for these set of points be y = f(x), the criterion under which the eqaution of the curve will be the best fit possible as follows,

The error for any point 'i' can be given by the function:

`Error = |y(i) - value of f(x) at x(i)|`

To remove the modulus sign, it is effective if we square the error term, so the square of all the n points is given by the summation of all squares,

`SSE = sumError(i)^2`

assume the proposed equation is a nth degree polynomial of the form,

f(x) = A1 + A2.x + A3.x^2+...... + An.x^(xn-1)

so when we differentiate the SSE with respect to its coefficients and equate it to zero, we can find the minima and the value of that constant. This method is called as least square method and is most commanly used technique as long as the fitting function is a polynomial.

What is a best fit?

A line is said to be best fit when distance between the raw data points and the predicted points are minimum, i.e. the difference between the raw data points and the predicted points is minimum.

There are 4 quantities which completely defines the goodness of the fit. They are:

1. Sum of Square of Errors
2. R-Square
3. Adjusted R-Square
4. Root Mean Square Error

1. Sum of Square of Errors as earlier defined is the summation of the square of the difference between the raw data points and predicted data points.

2. R-Square:

It is defined as the quantity that defines how successful the fit is in explaning the variation of the data. it is also called as the square of the multiple correlation cofficient and the coefficient of multiple determination.

The value of R-square actually defines how better the fit is, i.e. it's value ranges between the 0 and 1.

0 is the worst possible fit for given data set.

1 is the best possible fit for the given data set.

The value of R-Square can be calculated by using the formula:

R^2 = SSR/SST

SSR = Sum of square of Regression, it is basically defined as the square of the difference between the value of the fitting fuction and the arithmatic mean of the data set.

If y = f(x) is the fit curve then the SSR is defined as follows,

SSR=∑(f(x(i))−Mean)2

SST = SSR + SSE (SST is defined as the Sum of Square of Total)

The R-Square value can be misleading, when someone accesses for the goodness of the fit for linear regression analysis. The R-square has some problems so the adjusted R-square and predicted R-square value are designed to address.

Problem1: Every time when you add a predictor to the model, the value of R-square increases, it never decreases. Consequently a model with number of terms may appear to be better fit simply because it has more number of terms.

Problem2: If the model has too many predictor and higher order polynomials, it begins to model the random noise in the data. This condition is known as the overfitting of the model and it produces misleading high R-square values and is lessened ability to make predictions.

3. Adjusted R-Square:

Adjusted R-Square is a nomalizing variation of R-Square which neglects the variation of R-square due to insignificant independent values. It compare the explanatory powers of regression models that contains different number of predictors.

The adjusted R-square is a modified version of R-square that has been adjusted for the number of predictor in the model. The adjusted R-square increases only if the new term improves the model more than it would be expected to change. It decreases when the predictor improves the model by less than expected by change. The adjusted R-square can be negative, but its usually not. It is always lower than R-square.

In the simplified Best Subsets Regression output below, you can see where the adjusted R-squared peaks, and then declines. Meanwhile, the R-squared continues to increase.

4. Root mean Square Error:

It is a frequently used measure of the difference between the values predicted by the model or an estimator and the value observed. It represents the square root of the second sample moment of the differences between the predicted values and the observed values or the quadratic mean of these differences. These deviations are also knowns as residuals when the calculations are performed over the data sample that was used for estimation and are called as errors when computed out of sample. It is always a non-negative number and a value of 0 would indicate a perfect fit to the data. In general lower the root mean square value better the fit.

RMSE=√SSE/n

where n is the total number of data points.

Usually, when the value of R-square is greater than 0.95, then the fit is assumed to be good.

MATLAB CODE:

clear all
close all
clc

cp_data = load('data')

temperature = cp_data(:,1);
cp = cp_data(:,2);

%Linear Curve Fit
coeff = polyfit(temperature,cp,1)
predicted_cp = polyval(coeff,temperature);

%Sum of Square of Errors
Error = predicted_cp(:,1) - cp(:,1);
SSE = sum(Error.^2)

%R1^2 Value
SSR = sum((predicted_cp(:,1) - mean(cp(:,1))).^2)
SST = SSR + SSE

R1_square = SSR/SST;

if R1_square>0.95
    disp('fit is good')
else 
    disp('fit is not good')
end

hold on
plot(temperature,cp)
plot(temperature,predicted_cp)
xlabel('Temperature')
ylabel('Specific Heat')
grid on
title('Temperature VS Specific Heat')
legend('Raw Data','Linear Curve Fit')

%Cubic Polynomial Fit
coeff1 = polyfit(temperature,cp,3);
predicted_cp1 = polyval(coeff1,temperature);

%Sum of square of errors
Error1 = cp(:,1) - predicted_cp1(:,1)
SSE1 = sum(Error1.^2)

%R2^2 Value

SSR1 = sum((predicted_cp1(:,1) - mean(cp(:,1))).^2)
SST1 = SSR + SSE
R2_square = SSR1/SST1

if R2_square>0.95
    disp('fit is good')
else
    disp('fit is not good')
end


figure(2)
hold on
plot(temperature,cp)
plot(temperature,predicted_cp1)
grid on
legend('Raw data','Cubic Fit Curve')
xlabel('Temperature')
ylabel('Specific Heat')
title('Temperature VS Specific Heat')

Linear Curve Fit:

Cubic Curve Fit:

It was observed in Linear Curve Fit that the value of R-square was less than 0.95, hence it as not a good fit.

Similarly, in the Cubic Curve Fit, it was observed that the value of R-square was greater than 0.95, hence it was a good fit.