Curve fitting
Introduction to Curve Fitting:
Curve fitting is the process of constructing a curve, or mathematical function, that has the best fit to a series of data points, possibly subject to constraints. Curve fitting can involve either interpolation, where an exact fit to the data is required, or smoothing, in which a "smooth" function is constructed that approximately fits the data. A related topic is regression analysis, which focuses more on questions of statistical inference such as how much uncertainty is present in a curve that is fit to data observed with random errors. Fitted curves can be used as an aid for data visualization, to infer values of a function where no data are available, and to summarize the relationships among two or more variables. Extrapolation refers to the use of a fitted curve beyond the range of the observed data, and is subject to a degree of uncertainty since it may reflect the method used to construct the curve as much as it reflects the observed data.
Types of Curve fits:
The first degree polynomial equation
Y = ax + b
is a line with slope a. A line will connect any two points, so a first degree polynomial equation is an exact fit through any two points with distinct x coordinates.
If the order of the equation is increased to a second degree polynomial, the following results:
Y = ax2 + bx + c
This will exactly fit a simple curve to three points.
If the order of the equation is increased to a third degree polynomial, the following is obtained:
Y = ax3+ bx2 + cx + d
This will exactly fit four points.
Polynomial curves fitting points generated with a sine function. Red line is a first degree polynomial, green line is second degree, orange line is third degree and blue is fourth degree.
Thus from above image it can be seen that the higher the order of polynomial, the higher the number of points to fit the curve, which increases the flexibility of the curve to be fitted(more perfect fit).
AIM:
To write a MATLAB code to fit the Cp data, and to plot the linear and cubic fit curves.
Evaluating the goodness of Fit:
As is common in statistical literature, the term goodness of fit is used here in several senses: A “good fit” might be a model
- that the data could reasonably have come from, given the assumptions of least-squares fitting
- in which the model coefficients can be estimated with little uncertainty
- that explains a high proportion of the variability in your data, and is able to predict new observations with high certainty
Goodness-of-Fit Statistics
After using graphical methods to evaluate the goodness of fit, you should examine the goodness-of-fit statistics. Curve Fitting Toolbox™ software supports these goodness-of-fit statistics for parametric models:
- The sum of squares due to error (SSE)
- R-square
- Adjusted R-square
- Root mean squared error (RMSE)
Sum of Squares Due to Error(SSE): This statistic measures the total deviation of the response values from the fit to the response values. It is also called the summed square of residuals and is usually labelled as SSE.
A value closer to 0 indicates a better fit.
R-squared value(R^2):
R-squared is a statistical measure of how close the data are to the fitted regression line. It is also known as the coefficient of determination, or the coefficient of multiple determination for multiple regression.
The definition of R-squared is fairly straight-forward; it is the percentage of the response variable variation that is explained by a linear model. Or:
R-squared = Explained variation / Total variation
R-square is defined as the ratio of the sum of squares of the regression (SSR) and the total sum of squares (SST). SSR is defined as
SST is also called the sum of squares about the mean, and is defined as
where SST = SSR + SSE. Given these definitions, R-square is expressed as
R-square can take on any value between 0 and 1, with a value closer to 1 indicating that a greater proportion of variance is accounted for by the model. For example, an R-square value of 0.8234 means that the fit explains 82.34% of the total variation in the data about the average
R-squared is always between 0 and 100%:
- 0% indicates that the model explains none of the variability of the response data around its mean.
- 100% indicates that the model explains all the variability of the response data around its mean.
In general, the higher the R-squared, the better the model fits your data.
residual = data - fit
Perfect Fit and Best Fit:
Generally, curve fit algorithms determine the best-fit parameters by minimizing a chosen merit function. In order to optimize the merit function, it is necessary to select a set of initial parameter estimates and then iteratively refine the merit parameters until the merit function does not change significantly between iterations. The Levenberg-Marquardt algorithm has been used for nonlinear least squares calculations in the current implementation.
The goodness of fit is shown as an R^2-value. A value of R^2=1.0 indicates a perfect fit, whereas R^2=0.0 indicates that the regression model might be unsuitable for this type of data.
A best fit curve implies that, a curve that has the best fit to a series of data points, possibly subject to constraints. This curve fit need not fit all the data points as in Perfect fit but can deviate from a few data points based on the accuracy.
In general curve fitting problems a best fit to a higher accuracy is preferred rather than a perfect fit so that the curve fitting problem doesnot 'Overfit the data' and also donot overly complicate the problem.
Degrees of Freedom Adjusted R-Square:
The adjusted R-squared is a modified version of R-squared that has been adjusted for the number of predictors in the model. The adjusted R-squared increases only if the new term improves the model more than would be expected by chance. It decreases when a predictor improves the model by less than expected by chance. The adjusted R-squared can be negative, but it’s usually not. It is always lower than the R-squared.
The adjusted R-square statistic is generally the best indicator of the fit quality when you add additional coefficients to your model.
Root Mean Square:
This statistic is also known as the fit standard error and the standard error of the regression. It is an estimate of the standard deviation of the random component in the data, and is defined as
where MSE is the mean square error or the residual mean square
Just as with SSE, an MSE value closer to 0 indicates a fit that is more useful for prediction.
PROCEDURE:
1) Load the cp data file onto MATLAB using the 'load' command.
2) Import temperature and cp values separately as a column vector.
3) Now in first case we shall use a curve fit using first order polynomial. This can be done using the 'polyfit' and 'polyval'command.
4) Polyfit is a Matlab function that computes a least squares polynomial for a given set of data. Polyfit generates the coefficients of the polynomial, which can be used to model a curve to fit the data. Polyval evaluates a polynomial for a given set of x values.
5) Now the above same steps are repeated for different order polynomials.
6)The labels, legend and title of the plots are given using the 'xlabel','ylabel','title','legend' commands.
7) The goodness of fit can be found for each order plot,by evaluating the error values of each plot. The equations of SSE, SSR, RMS, R_sq_adj,R_sq,SST, mean from above sub heading, are then used to calculate and evaluate the goodness of fit.
8) In case of plotting splitwise polynomial curve, we have to divide the dataset into sub intervals.
9) Then each sub interval is plotted separately by a cubic polynomial. The Plot command is used at the end of each sub interval.
clear all
close all
clc
%preparing the data
cp_data = load('data');
temperature = cp_data(:,1);
cp = cp_data(:,2);
%curve fit for 1 degree polynomial
%cp = a*t + b
co_effs1 = polyfit(temperature,cp,1);
predicted_cp1 = polyval(co_effs1,temperature);
%curve fit for 2 degree polynomial
%cp = a*t^2 + b*t + c
co_effs2 = polyfit(temperature,cp,2);
predicted_cp2 = polyval(co_effs2,temperature);
%curve fit for 3 degree polynomial
%cp = a*t^3 + b*t^2 + ct + d
[co_effs3,s,mu] = polyfit(temperature,cp,3);
predicted_cp3 = polyval(co_effs3,temperature,s,mu);
%compare my curve fit for 1 degree polynomial with the orignal data.
figure(1)
plot (temperature,cp,'linewidth',3)
hold on
plot(temperature,predicted_cp1,'linewidth',3,'color','r')
xlabel('temperature [k]')
ylabel('specific heat [kJ/kmol-k]')
legend('original dataset curve ','predicted cp1 curve')
title('1 degree polynomial');
dim = [0.2 0.75 .1 .1];
str = ['Y = at + b'];
annotation('textbox',dim,'String',str,'FitBoxToText','on');
%compare my curve fit for 2 degree polynomial with the orignal data.
figure(2)
plot (temperature,cp,'linewidth',3)
hold on
plot(temperature,predicted_cp2,'linewidth',3,'color','r')
xlabel('temperature [k]')
ylabel('specific heat [kJ/kmol-k]')
legend('original dataset curve','predicted cp2 curve')
title('2 degree polynomial')
dim = [0.2 0.75 .1 .1];
str = ['Y = at^2 + bt + c'];
annotation('textbox',dim,'String',str,'FitBoxToText','on');
%compare my curve fit for 3 degree polynomial with the orignal data
figure(3)
plot (temperature,cp,'linewidth',3)
hold on
plot(temperature,predicted_cp3,'linewidth',3,'color','r')
xlabel('temperature [k]')
ylabel('specific heat [kJ/kmol-k]')
legend('original dataset curve','predicted cp3 curve')
title('3 degree polynomial')
dim = [0.2 0.75 .1 .1];
str = ['Y = at^3 + bt^2 + ct + d'];
annotation('textbox',dim,'String',str,'FitBoxToText','on');
%==========Evaluating goodness of fit for linear polynomial(first order) ==============%
%Calculating the SSE, SST, R_sq, R_sq_adj for linear Polynomial
Error1 = cp-predicted_cp1;
% Calculating SSE1
sse = 0;
sum1 = 0;
for i=1:length(cp)
SSE1 = sse+Error1(i)^2;
SUM1 = sum1+cp(i);
end
n = length(cp)
k = 2
mean = SUM1/n % Mean value of cp data
%Calculating SSR1
ssr= 0;
for j=1:length(predicted_cp1)
SSR1 = ssr+(predicted_cp1(i)-mean)^2;
end
%Calculating SST1
SST1 = SSR1+SSE1
% Calculating R_square value for linear polynomial
R_sq1 = SSR1/SST1
% Calculating Adjusted R_square value for linear polynomial
R_sq_adj1 = 1-((1-R_sq1)*(n-1)/(n-k-1))
%Calculating Root Mean Square of linear polynomial
RMSE1 = (SSE1/n)^0.5
%==========Evaluating goodness of fit for cubic polynomial(third order) ==============%
%Calculating the SSR, SST, R_sq, R_sq_adj for cubic Polynomial
Error3 = cp-predicted_cp3;
% Calculating SSE3
sse = 0;
sum3 = 0;
for i=1:length(cp)
SSE3 = sse+Error3(i)^2;
SUM3 = sum3+cp(i);
end
n = length(cp) % number of data points
k = 2 % number of independent variables
mean = SUM3/n % Mean value of cp data
%Calculating SSR3
ssr = 0;
for j=1:length(predicted_cp3)
SSR3 = ssr+(predicted_cp3(i)-mean)^2;
end
%Calculating SST3
SST3 = SSR3+SSE3
% Calculating R_square value for cubic polynomial
R_sq3 = SSR3/SST3
% Calculating Adjusted R_square value for cubic polynomial
R_sq_adj3 = 1-((1-R_sq3)*(n-1)/(n-k-1))
%Calculating Root Mean Square of cubic polynomial
RMSE3 = (SSE3/n)^0.5
Split wise curve fitting
This is also similar to a normal curve fitting problem but instead of using a single polynomial equation for the entire curve fit, we can use sub intervals of the data. And for each data a polynomial equation is given.
This helps in inceasing the flexibility with which we can control the 'curviness' or the 'fit' of the curve thereby increasing our chances of perfectly fitting the given data set. THe below image shows a splitwise curve fit for a bell-curve.
The below program is for the same cp data but the data is splitted into intervals and then fitted using cubic polynomials.
clc,close all,clear all
cp_data=load('data');
%Assign the values of the given data
temp=cp_data(:,1);
cp=cp_data(:,2);
% First Interval
temp1=cp_data(1:800,1);
cp1=cp_data(1:800,2);
%Second Interval
temp2=cp_data(801:1600,1);
cp2=cp_data(801:1600,2);
% Third Interval
temp3=cp_data(1601:2400,1);
cp3=cp_data(1601:2400,2);
%Fourth Interval
temp4=cp_data(2401:3200,1);
cp4=cp_data(2401:3200,2);
%find the coefficients,
[cp_coeff1,s1,mu1]=polyfit(temp1,cp1,3);
predicted_cp1=polyval(cp_coeff1,temp1,s1,mu1);
[cp_coeff2,s2,mu2]=polyfit(temp2,cp2,3);
predicted_cp2=polyval(cp_coeff2,temp2,s2,mu2);
[cp_coeff3,s3,mu3]=polyfit(temp3,cp3,3);
predicted_cp3=polyval(cp_coeff3,temp3,s3,mu3);
[cp_coeff4,s4,mu4]=polyfit(temp4,cp4,3);
predicted_cp4=polyval(cp_coeff4,temp4,s4,mu4);
%curve fit
plot(temp,cp,'linewidth',2.5)
hold on
%Plotting the first interval
plot(temp1,predicted_cp1,'color','r','linewidth',2.5)
%Plotting the second interval
plot(temp2,predicted_cp2,'color','g','linewidth',2.5)
%Plotting the third interval
plot(temp3,predicted_cp3,'color','y','linewidth',2.5)
%Plotting the fourth interval
plot(temp4,predicted_cp4,'color','m','linewidth',2.5)
hold off
xlabel('Temperature [K]')
ylabel('Specific Heat [KJ/Kmol-K]')
title('Splitwise Curve Fit Data for a Cubic Curve(3rd order)')
axis([250 3800 900 1500])
legend('Original Data','1st Split Curve Fit','2nd Split Curve Fit ','3rd Split Curve Fit ','4th Split Curve Fit ','location','northwest')Errors faced while programming:
1) The error is as follows,
While calculating the error values of the curve fit, in FOR loop of sse, failed to initialise the initial value of sse.
The error was overcome by assigning a initial value of zero.
2) The error warning is as shown below
This error was due to the fact that while using polyfit command. "Badly conditioned" means that the solution of the system of linear equations critically depends on rounding errors due to the limited precision.
Thus to overcome this warning, a solution is 'structuring' and 'scaling'. By adding s and mu after the polyval command the warning was overcome.The 'mu' scales the input x such that it has the mean of 0 and a standard deviation of 1. Since the errors involved in fitting this simple data set is not significant the warning is not that important.
But when using complex data set and curves using s and mu will be much valuble.
Output:
From the above codes the following outputs were found:
From the above graphs the following points can be inferred,
- The predicted cp curve fit is much more efficient in 3 degree polynomial plot compared to the first and second degree plot.
- THe main reason for the better fit of the third plot is that since there were three polynomial constants a, b, c in the equation, it helped us to contol the curve more flexibly as compared to curves with lesser constants.
- Thus it can be seen that the curve can be still made to fit perfectly just by using higher order polynomials, but on the contrary choosing a higher order polynomial always is not prefereable because of the following points,
- The curve tends to 'overfit' the data which works well only for the particular dataset which we have chosen, and has a higher chance of giving false predictions in some cases.
- It can over complicate the problem and takes more time to solve, in case of complex problems.
The same data set but with higher order polynomials was also done and the output is as follows,
Thus it can be seen that a more pefect fit curve is obtained just by using higher order polynomials.
The output of the splitwise polynomial is as below,
A better fit to the same cubic curve can be obtained by splitting the dataset into equal or unequal intervals and plotting each curve for the intervals.
The main advantage is that since the entire dataset is splitted into multiple sub intervals, each sub curve becomes less complicated. Hence the fit can be better achieved.
Curve fitting Toolbox:
The curve fitting toolbox by MATLAB helps us predict the values of the errors and helps us make a curve fit based on the polynomials and input functions. The above results and fit was verified using curve fittting toolbox.
Linear polynomial
Cubic polynomial
Higher order polynomial(9th order)
Similar plots were obtained in the MATLAB curve fitting toolbox, when compared the previous plots from code.
References:
1) https://in.mathworks.com/matlabcentral/answers/88536-polyfit-polynomial-is-badly-conditioned
2) https://in.mathworks.com/help/matlab/ref/polyfit.html
3) https://in.mathworks.com/help/curvefit/evaluating-goodness-of-fit.html#bq_5kwr-3
4)https://docs.tibco.com/pub/spotfire/6.5.0/doc/html/curve/curve_curve_fit_theory.htm