Open In Colab

Fitting a changing trend

We will use the time series of the log daily page views for the Wikipedia page for Peyton Manning as an example to illustrate how to fit a changing trend.

First, we load the data:

[1]:
if 'google.colab' in str(get_ipython()):
    !pip install git+https://github.com/ourownstory/neural_prophet.git # may take a while
    #!pip install neuralprophet # much faster, but may not have the latest upgrades/bugfixes

import pandas as pd
from neuralprophet import NeuralProphet, set_log_level
set_log_level("ERROR")
[2]:
data_location = "https://raw.githubusercontent.com/ourownstory/neuralprophet-data/main/datasets/"
df = pd.read_csv(data_location + "wp_log_peyton_manning.csv")
df.head(3)
[2]:
ds y
0 2007-12-10 9.590761
1 2007-12-11 8.519590
2 2007-12-12 8.183677

Now we can fit an initial model without any customizations.

We specify the data frequency to be daily. The model will remember this later when we predict into the future.

[3]:
m = NeuralProphet()
metrics = m.fit(df, freq="D")
metrics.head(3)

[3]:
SmoothL1Loss MAE MSE RegLoss
0 0.822911 5.544816 51.448767 0.0
1 0.368448 3.228539 18.241307 0.0
2 0.094250 1.527929 3.863379 0.0

The returned metrics dataframe contains recoded metrics for each training epoch.

Note: ‘y’ and ‘y_scaled’ are not given for the period extending into the future, as we do not know their true values.

[5]:
forecast = m.predict(df)
print(list(forecast.columns))
['ds', 'y', 'yhat1', 'residual1', 'trend', 'season_yearly', 'season_weekly']

The returned forecast dataframe contains the original datestamps, ‘y’ values, the predicted ‘yhat’ values, residuals and all the individual model components.

[6]:
# plots the model predictions
fig1 = m.plot(forecast)
_images/trend_peyton_manning_10_0.png
[7]:
# plots the individual forecast components for the given time period.
# fig = m.plot_components(forecast, residuals=True)
[8]:
# visualizes the model parameters.
fig2 = m.plot_parameters()
_images/trend_peyton_manning_12_0.png

Adjusting Trend

The default values work fairly well in this example. However, the default of 5 changepoints may not be adequate if the actual change in trend happens to fall in a region between the points.

Increasing Trend Flexibility

We can address this by increasing the number of changepoints, giving the trend more flexibility, at the danger of overfitting.

Let’s try what happens if we increase the number of changepoints to 30. Additionally, we can increase the range of data on which we fit trend changepoints to only exlude the last 10 percent (default is 20 percent).

Further, we can limit the trend flexibility by regularizing it.

[13]:
m = NeuralProphet(
    n_changepoints=30,
    changepoints_range=0.90,
    trend_reg=1,
)
metrics = m.fit(df, freq="D")
forecast = m.predict(df)

[14]:
fig1 = m.plot(forecast)
fig2 = m.plot_parameters()
_images/trend_peyton_manning_15_0.png
_images/trend_peyton_manning_15_1.png

Looking at the trend rate changes it becomes evident that the trend is overfitting to short-term fluctuations.

Manual Trend Changepoints

You can also manually specify the trend changepoints.

Note: A changepoint will always be added at the beginning. You can ignore it.

[11]:
m = NeuralProphet(
    changepoints=['2012-01-01', '2014-01-01'],
)
metrics = m.fit(df, freq="D")
forecast = m.predict(df)

[12]:
fig1 = m.plot(forecast)
fig2 = m.plot_parameters()
_images/trend_peyton_manning_19_0.png
_images/trend_peyton_manning_19_1.png
[ ]: