Tutorial 8: Uncertainty

NeuralProphet does support multiple ways to incorporate uncertainty into the forecast. In this tutorial, we will explore the quantile regression uncertainty feature. You can read more on Quantile regression on Wikipedia.

We start with the NeuralProphet base model from the previous tutorials and look at how to add uncertainty modelling to the forecast.

import pandas as pd
from neuralprophet import NeuralProphet, set_log_level

# Disable logging messages unless there is an error
set_log_level("ERROR")

# Load the dataset from the CSV file using pandas
df = pd.read_csv("https://github.com/ourownstory/neuralprophet-data/raw/main/kaggle-energy/datasets/tutorial01.csv")

# Model and prediction
m = NeuralProphet()
m.set_plotting_backend("plotly-static")
metrics = m.fit(df)
forecast = m.predict(df)
m.plot(forecast)

In comparison to point forecasting, uncertainty modelling predicts an interval of possible values for the forecast. Assuming we want the true value to be within the estimated interval with a probability of 90%, we set the confidence level to be 0.9. We define two quantiles, 0.05 and 0.95, which correspond to the 5th and 95th percentiles of the distribution of the forecast. The 90% confidence interval is then the difference between the 5th and 95th percentiles.

confidence_level = 0.9

boundaries = round((1 - confidence_level) / 2, 2)
# NeuralProphet only accepts quantiles value in between 0 and 1
quantiles = [boundaries, confidence_level + boundaries]

Setup model with uncertainty using the 10th and 90th percentiles.

# Create NeuralProphet model with list of quantiles
m = NeuralProphet(quantiles=quantiles)
m.set_plotting_backend("plotly-static")

metrics = m.fit(df)
forecast = m.predict(df)
m.plot(forecast)

For the above method we used a quantile regression model, that uses the pinball loss function to assess the goodness-of-fit. While the defined coverage is guaranteed for the training date, it is not guaranteed for unseen data. More complex methods like Conformal Prediction can be used to guarantee marginal coverage for unseen data:

# evaluate uncertainy on calibration set
train_df, cal_df = m.split_df(df, valid_p=0.1)
method = "naive"  # or "cqr" for a more sophisticated method, see uncertainty tutorial
conformal_forecast = m.conformal_predict(train_df, cal_df, alpha=0.1, method=method)
m.highlight_nth_step_ahead_of_each_forecast(1).plot(conformal_forecast)

For a deeper dive into the topic, you can check out our tutorial on Uncertainty Quantification.