Polynomial Curve Fitting Using MATLAB
A MATLAB code has been written to fit a set of data points on the variation of Cp (Specific Heat) with Temperature, using linear and Cubic polynomials. The linear and cubic fit curves are then evaluated based on their fitness characteristics and are also compared on the same plot, along with the raw data points.
Intially we load the raw data from the data file into MATLAB using the "load('data') command, then we assign variables to hold all the values of Cp and all the values of temperature.
We try to fit the curve in such a way that the error between the curve fir and the actual curve is very minimal.
Initially while fitting the curve, we make assumptions on the type of polynomial that would required to fit the entire curve or atleast a section of the curve.
For exmaple, if we assume a linear polynomial for the fit. It'll be of the form y = ax^2 +bx + c.
For our data set, it will be Cp = aT^2 + bT + c
For a Cubic Polynomial, it will be Cp = aT^3 + bT^2 + cT + d
We then find the values of the coefficeint using the points from the data set.
In MATLAB we can do this using the Polynomial curve fitting command, polyfit(x,y,n)
p = polyfit(x,y,n) returns the coefficients for a polynomial p(x) of degree n that is a best fit (in a least-squares sense) for the data in y. The coefficients in p are in descending powers, and the length of p is n+1
p(x)=p1xn+p2xn−1+...+pnx+pn+1
We can then use the polyval(polyfit coefficients,x) command, that uses the coefficients obtained from then polyfit command and the x(temperature) values from the data set to obtain the predicted y(Cp) values.
y = polyval(p,x) returns the value of a polynomial of degree n evaluated at x. The input argument p is a vector of length n+1 whose elements are the coefficients in descending powers of the polynomial to be evaluated.
x can be a matrix or a vector. In either case, polyval evaluates p at each element of x.
After obtaining the predicted values of the curve fit, using a linear or cubic polynomial. We can then plot them against the raw data points and evaluate their fitness characteristics.
Increasing the order of the ploynomials makes the the fit better, but the polynomials become badly conditioned.
There are several reasons given to get an approximate fit when it is possible to simply increase the degree of the polynomial equation and get an exact match.:
- Even if an exact match exists, it does not necessarily follow that it can be readily discovered. Depending on the algorithm used there may be a divergent case, where the exact fit cannot be calculated, or it might take too much computer time to find the solution. This situation might require an approximate solution.
- The effect of averaging out questionable data points in a sample, rather than distorting the curve to fit them exactly, may be desirable.
- Runge's phenomenon: high order polynomials can be highly oscillatory. If a curve runs through two points A and B, it would be expected that the curve would run somewhat near the midpoint of A and B, as well. This may not happen with high-order polynomial curves; they may even have values that are very large in positive or negative magnitude. With low-order polynomials, the curve is more likely to fall near the midpoint (it's even guaranteed to exactly run through the midpoint on a first-degree polynomial).
- Low-order polynomials tend to be smooth and high order polynomial curves tend to be "lumpy". To define this more precisely, the maximum number of inflection points possible in a polynomial curve is n-2, where n is the order of the polynomial equation. An inflection point is a location on the curve where it switches from a positive radius to negative. We can also say this is where it transitions from "holding water" to "shedding water". Note that it is only "possible" that high order polynomials will be lumpy; they could also be smooth, but there is no guarantee of this, unlike with low order polynomial curves. A fifteenth-degree polynomial could have, at most, thirteen inflection points, but could also have twelve, eleven, or any number down to zero.
The degree of the polynomial curve being higher than needed for an exact fit is undesirable for all the reasons listed previously for high order polynomials, but also leads to a case where there are an infinite number of solutions. For example, a first-degree polynomial (a line) constrained by only a single point, instead of the usual two, would give an infinite number of solutions. This brings up the problem of how to compare and choose just one solution, which can be a problem for software and for humans, as well. For this reason, it is usually best to choose as low a degree as possible for an exact match on all constraints, and perhaps an even lower degree, if an approximate fit is acceptable.
To help with this and find a better way to fit the data, we can use splitwise polynomials, meaning we can use 1 polynomial set for 1 temperauture range and another polynomial for a different temperature range.
Code:
clear all
close all
clc
%Preparing the data
cp_data = load('data');
temperature = cp_data(:,1);
cp = cp_data(:,2);
%Curve Fit, Linear Poynomial
% Cp = a*T^2 + b*T + c
co_effs_2 = polyfit(temperature,cp,2);
predicted_cp_2 = polyval(co_effs_2,temperature);
%Curve Fit, Cubic Poynomial
% Cp = a*T^3 + b*T^2 + c*T + d
co_effs_3 = polyfit(temperature,cp,3);
predicted_cp_3 = polyval(co_effs_3,temperature);
%Compare my curve fit with the original data
%Plot
plot(temperature,cp,'linewidth',3,'color','b')
hold on
plot(temperature,predicted_cp_2,'linewidth',3,'color','r')
plot(temperature,predicted_cp_3,'linewidth',3,'color','g')
xlabel('Temperature [K]')
ylabel('Specific Heat [KJ/Kmol.K]')
legend('Original Data set','Linear Polynomial Predicted Data Set','Cubic Polynomial Predicted Data Set')Error faced:
Not using the hold on command. This provided only the last figure that is asked by MATLAB to be displayed and not the ones asked before it.
This was fixed using the hold on command.
Results:
Resulting Plot comparing the curve fit and the raw data set:
There are 4 quantities which help us in measuring the goodness of fit criteria or how well the equation is a representing the datapoints. They are:
- The sum of squares due to error (SSE)
- R-square
- Adjusted R-square
- Root mean squared error (RMSE)
1.) Sum of Squares Due to Error
This statistic measures the total deviation of the response values from the fit to the response values. It is also called the summed square of residuals and is usually labelled as SSE.
A value closer to 0 indicates that the model has a smaller random error component, and that the fit will be more useful for prediction.
2.) R-Square
This statistic measures how successful the fit is in explaining the variation of the data. Put another way, R-square is the square of the correlation between the response values and the predicted response values. It is also called the square of the multiple correlation coefficient and the coefficient of multiple determination.
R-square is defined as the ratio of the sum of squares of the regression (SSR) and the total sum of squares (SST). SSR is defined as:
R-square can take on any value between 0 and 1, with a value closer to 1 indicating that a greater proportion of variance is accounted for by the model. For example, an R-square value of 0.8234 means that the fit explains 82.34% of the total variation in the data about the average.
If you increase the number of fitted coefficients in your model, R-square will increase although the fit may not improve in a practical sense. To avoid this situation, you should use the degrees of freedom adjusted R-square statistic described below.
Note that it is possible to get a negative R-square for equations that do not contain a constant term. Because R-square is defined as the proportion of variance explained by the fit, if the fit is actually worse than just fitting a horizontal line then R-square is negative. In this case, R-square cannot be interpreted as the square of a correlation. Such situations indicate that a constant term should be added to the model.
3.) Degrees of Freedom Adjusted R-Square
This statistic uses the R-square statistic defined above, and adjusts it based on the residual degrees of freedom. The residual degrees of freedom is defined as the number of response values n minus the number of fitted coefficients m estimated from the response values.
The adjusted R-square statistic can take on any value less than or equal to 1, with a value closer to 1 indicating a better fit. Negative values can occur when the model contains terms that do not help to predict the response.
4.) Root Mean Squared Error
This statistic is also known as the fit standard error and the standard error of the regression. It is an estimate of the standard deviation of the random component in the data, and is defined as:
Just as with SSE, an MSE value closer to 0 indicates a fit that is more useful for prediction. Therefore an RSME value closer to 0 indicates a fit that is more useful for prediction.
Using these four quantities Matlab effectively deduces the goodness of fit, typically if the R-square is greater than 0.95, the fit is assumed to be good.
The fitness characteristics for both the linear and cubic curves using the curve fitting toolbox in MATLAB:
The SSE value for the cubic fit is closer to 0 than the linear fit indicating that the model has a smaller random error component, and that the fit will be more useful for prediction.
The R-square value for the cubic fit is closer to 1 than the linear fit indicating that a greater proportion of variance is accounted for by the cubic fit model, i.e the fit explains 99.67% of the total variation in the data about the average.
The adjusted R-square statistic is generally the best indicator of the fit quality when you compare two models that are nested, — that is, a series of models each of which adds additional coefficients to the previous model, such as in our case of linear and cubic polynomial fits for the same data set. However the adjusted R-square value for the cubic fit is closer to 1 than the linear fit indicating a better fit for the cubic model.
The RSME value, which is an estimate of the standard deviation of the random component in the data, is closer to 0 for the cubic fit than the linear fit, indicating that the cubic fit would be more useful for prediction.
In summary, all the 4 characteristic quantitites used to measure the goodness of the fit criteria favour the cubic fit over the linear fit.
1.) How to make a curve fit perfectly?
There is no standard and or process to achieve a curve fit perfectly. Based on the raw data of a particular data set, we can only optimize a fit with by reducing the error values until, for a given data set, the model exactly fits all of the points in the data. For a perfect model the R - Square value will be equal to 1 and SSE should be 0.
2.)How to get the best fit? (Best fit and perfect fit are two completely different phenomena.)
So overall, there are only rules of thumb or preferences that we can follow to obtain the best fit for a particular data set. We can only fit the curve to such an extent that the error and fitness characteristics are within certain limits and are acceptable to a certain degree of correctness.
For example for data sets that can be predicted using a polynomial fit.
We might tend to overfit or underfit.
Overfit
Overfit - over-doing the requirement for the fit to ‘match’ the data trend (order too high).
Polynomials become more ‘squiggly’ as their order increases. A ‘squiggly’ appearance comes from inflections in function.
Considerations:
Consideration #1:
3rd order - 1 inflection point
4th order - 2 inflection points
nth order - n-2 inflection points
Consideration #2:
2 data points - linear touches each point
3 data points - second order touches each point
n data points - n-1 order polynomial will touch each point
SO: Picking an order too high will overfit data
General rule: pick a polynomial form at least several orders lower than the number of data points. Start with linear and add order until trends are matched.
Underfit
If the order is too low to capture obvious trends in the data.
General rule: View data first, then select an order that reflects inflections, etc.
For the example above:
1) Obviously nonlinear, so order > 1
2) No inflcetion points observed as obvious, so order < 3 is recommended
Therefore use 2nd order for this data
If polynomials aren't sufficient to fit the curves, we have to move onto exponential methods, or non-linear equation solving methods or linearization.
3.) What could be done to improve the cubic fit?
To improve a cubic fit, we can add weights to each and every term in the polynomial coefficients of the cubic fit. This would ensure that the effect of a particular term allows improving the accuracy of the fit. This can be done by a trial and error method.
References:
https://www.essie.ufl.edu/~kgurl/Classes/Lect3421/Fall_01/NM5_curve_f01.pdf
https://in.mathworks.com/help/curvefit/evaluating-goodness-of-fit.html
Sandra Lach Arlinghaus, PHB Practical Handbook of Curve Fitting. CRC Press, 1994.
https://www.hindawi.com/journals/jmath/2013/753981/
https://projects.skill-lync.com/projects/Curve-Fitting-and-criterion-to-choose-best-fit-41860