Renewable Energy: Forecasting hourly solar irradiance

The training will occur on 90% of the data, reserving the last 10% for evaluation.

[1]:
if 'google.colab' in str(get_ipython()):
    !pip install git+https://github.com/ourownstory/neural_prophet.git # may take a while
    #!pip install neuralprophet # much faster, but may not have the latest upgrades/bugfixes

import pandas as pd
import numpy as np
from neuralprophet import NeuralProphet, set_log_level
set_log_level("ERROR")
[2]:
# data_location = "https://raw.githubusercontent.com/ourownstory/neuralprophet-data/main/datasets/"
data_location = '../../../neuralprophet-data/datasets/'

sf_pv_df = pd.read_csv(data_location +  'energy/SF_PV.csv')

Generic forecast: Time-based features only

In this first section, we will train a model with time-features only like we would do with Facebook Prophet.

From the start, we know that the sun intensity depends on the time of day and season in the year. Further, we know that the daily peak intensity is proportional to the season. As of now, neither Prophet nor NeuralProphet can multiply two seasonalities, thus, the peak will bee to low in summer and may turn negative in winter. Not ideal, but ok for a first attempt.

[3]:
m = NeuralProphet(
    yearly_seasonality=3,
    weekly_seasonality=False,
    daily_seasonality=8,
    growth="off",
    learning_rate=0.1,
)
df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)

metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)
_images/energy_solar_pv_4_0.png

[4]:
metrics.tail(1)
[4]:
SmoothL1Loss MAE MSE RegLoss SmoothL1Loss_val MAE_val MSE_val
107 0.009197 92.454981 14308.832125 0.0 0.013244 127.825119 20605.609375
[5]:
forecast = m.predict(sf_pv_df)
fig = m.plot(forecast)
_images/energy_solar_pv_6_0.png
[6]:
fig_param = m.plot_parameters()

_images/energy_solar_pv_7_0.png
[7]:
forecast = m.predict(df_test)
m = m.highlight_nth_step_ahead_of_each_forecast(1)
fig = m.plot(forecast)
_images/energy_solar_pv_8_0.png
[8]:
fig_prediction = m.plot(forecast[-48:])

_images/energy_solar_pv_9_0.png
[ ]:

1-step ahead forecast with Auto-Regresseion

In this second section, we will train a 1-step ahead forecaster on solar irradiance data (that can be a proxy for solar PV production). We can train this forecaster considering the privious 24 steps and disabling trend and seasonality.

The paramters that we can modify are the number of lags and the AR sparsity.

[9]:
m = NeuralProphet(
    growth='off',
    yearly_seasonality=False,
    weekly_seasonality=False,
    daily_seasonality=False,
    n_lags=3*24,
    learning_rate=0.01,
)

df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)

metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)
_images/energy_solar_pv_12_0.png

[10]:
metrics.tail(1)
[10]:
SmoothL1Loss MAE MSE RegLoss SmoothL1Loss_val MAE_val MSE_val
107 0.001957 30.255167 3045.023082 0.0 0.000602 18.303713 935.846252
[11]:
forecast = m.predict(sf_pv_df)
fig = m.plot(forecast)
_images/energy_solar_pv_14_0.png
[12]:
forecast = m.predict(df_test)
m = m.highlight_nth_step_ahead_of_each_forecast(1)
fig = m.plot(forecast)
_images/energy_solar_pv_15_0.png
[13]:
fig_comp = m.plot_components(forecast)
_images/energy_solar_pv_16_0.png
[14]:
fig_param = m.plot_parameters()

_images/energy_solar_pv_17_0.png
[15]:
fig_prediction = m.plot(forecast[-48:])

_images/energy_solar_pv_18_0.png

Sparsifying the AR coefficients

By setting an ar_sparsity < 1 we can reduce the number of non-zero AR coefficients.

[16]:
m = NeuralProphet(
    growth='off',
    yearly_seasonality=False,
    weekly_seasonality=False,
    daily_seasonality=False,
    n_lags=3*24,
    ar_sparsity=0.8,
    learning_rate=0.01,
)

df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)

metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)
_images/energy_solar_pv_20_0.png

[17]:
metrics.tail(1)
[17]:
SmoothL1Loss MAE MSE RegLoss SmoothL1Loss_val MAE_val MSE_val
107 0.002148 32.992762 3342.54368 0.001164 0.00068 19.354525 1058.18103
[18]:
m = m.highlight_nth_step_ahead_of_each_forecast(1)
fig_param = m.plot_parameters()
_images/energy_solar_pv_22_0.png

1-step ahead forecast with Auto-Regression including Integration

Next, we will add the differences of the series as a lagged covariate. This basically extends the model from AR to ARI, where the I stands for ‘integrated’ time series.

[19]:
df = sf_pv_df.copy(deep=True)
df["I"] = np.append(0, sf_pv_df["y"].values[1:] - sf_pv_df["y"].values[:-1])
df.tail(3)
[19]:
ds y I
8757 2015-12-31 22:00:00 0 0
8758 2015-12-31 23:00:00 0 0
8759 2016-01-01 00:00:00 0 0
[20]:
m = NeuralProphet(
    growth='off',
    yearly_seasonality=False,
    weekly_seasonality=False,
    daily_seasonality=False,
    n_lags=3*24,
    learning_rate=0.01,
)
m = m.add_lagged_regressor("I", normalize="standardize")
df_train, df_test = m.split_df(df, freq='H', valid_p = 0.10)

metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)
_images/energy_solar_pv_25_0.png

[21]:
metrics.tail(1)
[21]:
SmoothL1Loss MAE MSE RegLoss SmoothL1Loss_val MAE_val MSE_val
107 0.001941 29.899686 3019.55232 0.0 0.000599 18.041283 931.748108
[22]:
m = m.highlight_nth_step_ahead_of_each_forecast(1)
fig_param = m.plot_parameters()
_images/energy_solar_pv_27_0.png

1 step ahead forecast with AR-Net: Using a Neural Network

There is something to consider here, if we consider a neural network with at least one hidden layer. For a high enough learning rate (probably > 0.1), the gradient seems to vanish and forces the AR net output to 0.

From below, we can see that the forecast output has a strange behavior. it looks like the output is exactly the sum of the weekly, daily seasonlality with trend. With no noise and changes at all.

[23]:
m = NeuralProphet(
    growth='off',
    yearly_seasonality=False,
    weekly_seasonality=False,
    daily_seasonality=False,
    n_lags=3*24,
    num_hidden_layers=4,
    d_hidden=16,
    learning_rate=1.0,
)

df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)

metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)
_images/energy_solar_pv_29_0.png

[25]:
fig_comp = m.plot_components(forecast)
_images/energy_solar_pv_30_0.png

Learning Rate matters when training a Neural Network.

An easy fix is to set the learning rate at a low enough value, likely around 0.01 to 0.001.

[26]:
m = NeuralProphet(
    growth='off',
    yearly_seasonality=False,
    weekly_seasonality=False,
    daily_seasonality=False,
    n_lags=3*24,
    num_hidden_layers=4,
    d_hidden=16,
    learning_rate=0.003,
)

df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)

metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)
_images/energy_solar_pv_32_0.png

[27]:
metrics.tail(1)
[27]:
SmoothL1Loss MAE MSE RegLoss SmoothL1Loss_val MAE_val MSE_val
107 0.001331 20.654574 2070.965949 0.0 0.00055 13.492601 855.404907
[28]:
fig = m.plot_parameters()
_images/energy_solar_pv_34_0.png
[29]:
m = m.highlight_nth_step_ahead_of_each_forecast(1)
forecast = m.predict(df_test)
fig = m.plot(forecast)
_images/energy_solar_pv_35_0.png
[30]:
fig_comp = m.plot_components(forecast)
_images/energy_solar_pv_36_0.png
[31]:
fig_prediction = m.plot(forecast[-48:])
_images/energy_solar_pv_37_0.png
[ ]: