To make a curve that perfectly fits the given data, we can create a polynomial.  An important question is how large our polynomial will be.  If we want a unique solution, there is only one proper size.  Think for a moment.  There is an infinite numbers of lines that can go through a single point, yet there is only a single line that can go through any two given points.  Similarly, a binomial can go through any three points.

In other words, a single constant value, will give you a horizontal line that can go through any single point.  If you have two unknown constants of the form y=ax+b, a line, you can go through any two points.  If you want to go through n-number of points, you need n-number of unknown constants.  Here is our equation.

We make this useful by substituting each point's x and y coordinates into the x and y values of the equation.  All 6 equations are below.

Six equations and six unknowns, sounds like something we can solve!  A great way that we can solve this problem is if we arrange it in matrix form.

To solve this, we want the variables by themselves.  To do that we take the inverse of the big left-most matrix, and multiply it to the left side of both parts of the equation.  Note that matrices are different from normal variables, and whether you multiply to the right side or to the left side matters. (i.e. matrix multiplication is not commutative)

Good lord that's horrible! Do the matrix multiplication out on the left side, and we simplify it all down to the identity matrix.  You can basically just erase that, it's the same thing as multiplying by 1 in matrices.  Now all we have to do is multiply out the right side using normal matrix multiplication...

And we have our answers!

Plugging them back into our original polynomial way up above, we have the following equation.

And here's our graph!  Note that this 5th order polynomial curve is unique, there's only one exact answer to fitting this curve with that polynomial.

Some of the more seasoned veterans will note that I picked points that make a kind of s-shaped x^3 graph.  Let's use least squares to fit a 3rd-order polynomial to the data.  First we create a polynomial just like before, but this time we use only 4 constants.  In fancy mathematical talk, this means we allow for a null space.

Just like before, we make 6 equations and plug in the coordinates of each point into each equation.

Line everything up, and put it into matrix form.  Unfortunately, there's no way to solve this!  We can't take the inverse of the left matrix because it isn't square!

Now here's the trick.  Since the null space is perpendicular to the transpose of the range (what??), we can multiply the whole thing by the transpose of the big matrix.

Doing all the matrix multiplication out, we now have a 4x4!  Rather than taking the summation of all the distances of all the points and minimizing them, we find the point that exists on the range that is closest to the null space.

Now all we do is take the inverse of the big matrix and multiply it to the left of both sides of the equation.

On the left it simplifies out to the identity matrix, which we can essentially drop.  On the right we have our constants!

Plugging those into our constants, we have our least squares 3rd order polynomial!

Here's our graph!  This is the closest-possible fitting 3rd order polynomial to the given data.

• Least Squares Fitting -- from MathWorld
• The Geometer's Sketchpad: Least Squares