Linear Regression vs Degree-2 Curve Fitting

```

What happens when our data rises, reaches a peak, and then falls? A straight regression line may completely miss the pattern. In this lesson, we compare linear regression with degree-2 polynomial curve fitting using the same dataset in both Python and JavaScript.

```

The Dataset

Suppose we record website traffic over seven consecutive time periods. Traffic starts low, rises toward the middle of the period, reaches a peak, and then falls again.

```
Time Period 1 2 3 4 5 6 7
Visitors 20 45 62 70 62 45 20
```

Why Linear Regression Struggles

Linear regression assumes that the relationship between x and y can be represented using a straight line:

y = mx + c

The first half of our data is increasing while the second half is decreasing. Because the dataset is almost perfectly symmetric, these opposite trends cancel each other when we calculate the best straight line.

The result is approximately:

Linear Regression:
y = 0x + 46.2857

In other words, the linear model effectively says that website traffic stays around 46 visitors throughout the entire period.

Clearly, that does not describe what is actually happening. The real data rises from 20 visitors to 70 visitors and then falls back to 20.

Degree-2 Curve Fitting

Instead of forcing the data into a straight line, we can allow the model to bend by introducing an x² term.

y = ax² + bx + c

We now need to calculate three unknown values: a, b, and c.

Instead of using a ready-made function such as numpy.polyfit(), our programs calculate the coefficients mathematically.

The Normal Equations

aΣx⁴ + bΣx³ + cΣx² = Σx²y
aΣx³ + bΣx² + cΣx = Σxy
aΣx² + bΣx + cn = Σy

These three simultaneous equations are converted into an augmented matrix. We then solve the matrix using Gaussian elimination followed by back substitution.

For our website-traffic dataset, the fitted equation is approximately:

Degree-2 Curve:
y = -5.3810x² + 43.0476x - 18.2857

Notice that the coefficient of x² is negative. This means the parabola opens downward. It can therefore rise, reach a maximum, and then fall — exactly the kind of behaviour visible in our data.

Comparing the Error

A visual graph already shows that the degree-2 curve fits much better, but we can measure the difference mathematically using Mean Squared Error (MSE).

```
Linear Regression
MSE ≈ 348.7755
Degree-2 Curve
MSE ≈ 1.3197
```

The difference is enormous. For this particular dataset, the quadratic model follows the observations much more closely than the straight-line model.

Try the Python Version

The Python program performs both calculations from basic mathematics. Linear regression is calculated from the standard least-squares equations, while the degree-2 curve is calculated using the normal equations and Gaussian elimination.

No polynomial fitting function is used. Matplotlib is used only to draw the final graph.

Try the JavaScript Version

The JavaScript version performs the same calculations directly in the browser. It uses ordinary JavaScript loops to calculate all the required sums, Gaussian elimination to solve the degree-2 normal equations, and an HTML <canvas> to draw the graph.

This is useful because students can see that the mathematics is the same regardless of whether we implement it in Python or JavaScript.

What Should We Learn From This?

The purpose of this example is not simply to show that a quadratic equation gives a smaller error. The more important lesson is that the shape of the model must match the shape of the data.

A straight line is excellent when the relationship is approximately linear. But when the data rises and then falls, a straight line has no mechanism for representing that turning point.

Adding the x² term gives the model enough flexibility to create one bend. That makes degree-2 curve fitting appropriate for many single rise-and-fall relationships.

Important distinction: this example looks cyclical because the values rise and fall, but a quadratic curve is not a truly periodic model. It can model one rise-and-fall shape. If the data repeatedly follows rise → fall → rise → fall, then models based on sine and cosine functions are more appropriate.

The Bigger Machine Learning Lesson

Machine learning is not simply about choosing an algorithm and asking it to produce an answer. We must first examine the behaviour of the data.

If the data follows a straight trend, linear regression may be enough. If the data contains curvature, polynomial terms may provide a better model. If the behaviour repeats periodically, we need a model that can represent periodic behaviour.

This small example therefore introduces one of the most important ideas in machine learning:

Do not force the data to fit the model.
Choose a model capable of representing the pattern in the data.

Learn With Champak
Learn the mathematics, understand the algorithm, and then write the code.