Renewable Energy: Forecasting hourly solar irradiance

The training will occur on 90% of the data, reserving the last 10% for evaluation.

[1]:
if 'google.colab' in str(get_ipython()):
    !pip install git+https://github.com/ourownstory/neural_prophet.git # may take a while
    #!pip install neuralprophet # much faster, but may not have the latest upgrades/bugfixes

import pandas as pd
import numpy as np
from neuralprophet import NeuralProphet, set_log_level
set_log_level("ERROR")
[2]:
# data_location = "https://raw.githubusercontent.com/ourownstory/neuralprophet-data/main/datasets/"
data_location = '../../../neuralprophet-data/datasets/'

sf_pv_df = pd.read_csv(data_location +  'energy/SF_PV.csv')

Generic forecast: Time-based features only

In this first section, we will train a model with time-features only like we would do with Facebook Prophet.

From the start, we know that the sun intensity depends on the time of day and season in the year. Further, we know that the daily peak intensity is proportional to the season. As of now, neither Prophet nor NeuralProphet can multiply two seasonalities, thus, the peak will bee to low in summer and may turn negative in winter. Not ideal, but ok for a first attempt.

[3]:
m = NeuralProphet(
    yearly_seasonality=3,
    weekly_seasonality=False,
    daily_seasonality=8,
    growth="off",
    learning_rate=0.1,
)
df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)

metrics = m.fit(df_train, freq='H', validation_df=df_test, progress='plot')
_images/energy_solar_pv_4_0.png
log-SmoothL1Loss
        training                 (min:   -4.688, max:   -0.279, cur:   -4.688)
        validation               (min:   -4.536, max:   -1.100, cur:   -4.273)
[4]:
metrics.tail(1)
[4]:
SmoothL1Loss MAE RMSE RegLoss SmoothL1Loss_val MAE_val RMSE_val
108 0.009209 92.148668 118.048201 0.0 0.013935 131.747787 147.242722
[5]:
forecast = m.predict(sf_pv_df)
fig = m.plot(forecast)
_images/energy_solar_pv_6_0.png
[6]:
fig_param = m.plot_parameters()

_images/energy_solar_pv_7_0.png
[7]:
forecast = m.predict(df_test)
m = m.highlight_nth_step_ahead_of_each_forecast(1)
fig = m.plot(forecast)
_images/energy_solar_pv_8_0.png
[8]:
fig_prediction = m.plot(forecast[-48:])

_images/energy_solar_pv_9_0.png
[ ]:

1-step ahead forecast with Auto-Regresseion

In this second section, we will train a 1-step ahead forecaster on solar irradiance data (that can be a proxy for solar PV production). We can train this forecaster considering the privious 24 steps and disabling trend and seasonality.

The paramters that we can modify are the number of lags and the AR sparsity.

[9]:
m = NeuralProphet(
    growth='off',
    yearly_seasonality=False,
    weekly_seasonality=False,
    daily_seasonality=False,
    n_lags=3*24,
    learning_rate=0.01,
)

df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)

metrics = m.fit(df_train, freq='H', validation_df=df_test, progress='plot')
_images/energy_solar_pv_12_0.png
log-SmoothL1Loss
        training                 (min:   -6.236, max:   -2.486, cur:   -6.236)
        validation               (min:   -7.432, max:   -3.793, cur:   -7.420)
[10]:
metrics.tail(1)
[10]:
SmoothL1Loss MAE RMSE RegLoss SmoothL1Loss_val MAE_val RMSE_val
108 0.001958 30.204243 52.720051 0.0 0.000599 18.234327 30.539005
[11]:
forecast = m.predict(sf_pv_df)
fig = m.plot(forecast)
_images/energy_solar_pv_14_0.png
[12]:
forecast = m.predict(df_test)
m = m.highlight_nth_step_ahead_of_each_forecast(1)
fig = m.plot(forecast)
_images/energy_solar_pv_15_0.png
[13]:
fig_comp = m.plot_components(forecast)
_images/energy_solar_pv_16_0.png
[14]:
fig_param = m.plot_parameters()

_images/energy_solar_pv_17_0.png
[15]:
fig_prediction = m.plot(forecast[-48:])

_images/energy_solar_pv_18_0.png

Sparsifying the AR coefficients

By setting an ar_reg > 0 we can reduce the number of non-zero AR coefficients.

[16]:
m = NeuralProphet(
    growth='off',
    yearly_seasonality=False,
    weekly_seasonality=False,
    daily_seasonality=False,
    n_lags=3*24,
    ar_reg=1,
    learning_rate=0.01,
)

df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)

metrics = m.fit(df_train, freq='H', validation_df=df_test, progress='plot')
_images/energy_solar_pv_20_0.png
log-SmoothL1Loss
        training                 (min:   -6.178, max:   -0.417, cur:   -6.157)
        validation               (min:   -7.415, max:   -0.230, cur:   -7.319)
[17]:
metrics.tail(1)
[17]:
SmoothL1Loss MAE RMSE RegLoss SmoothL1Loss_val MAE_val RMSE_val
108 0.002119 32.044212 55.196571 0.001024 0.000663 18.364031 32.107029
[18]:
m = m.highlight_nth_step_ahead_of_each_forecast(1)
fig_param = m.plot_parameters()
_images/energy_solar_pv_22_0.png

1-step ahead forecast with Auto-Regression including Integration

Next, we will add the differences of the series as a lagged covariate. This basically extends the model from AR to ARI, where the I stands for ‘integrated’ time series.

[19]:
df = sf_pv_df.copy(deep=True)
df["I"] = np.append(0, sf_pv_df["y"].values[1:] - sf_pv_df["y"].values[:-1])
df.tail(3)
[19]:
ds y I
8757 2015-12-31 22:00:00 0 0
8758 2015-12-31 23:00:00 0 0
8759 2016-01-01 00:00:00 0 0
[20]:
m = NeuralProphet(
    growth='off',
    yearly_seasonality=False,
    weekly_seasonality=False,
    daily_seasonality=False,
    n_lags=3*24,
    learning_rate=0.01,
)
m = m.add_lagged_regressor("I", normalize="standardize")
df_train, df_test = m.split_df(df, freq='H', valid_p = 0.10)

metrics = m.fit(df_train, freq='H', validation_df=df_test, progress='plot')
_images/energy_solar_pv_25_0.png
log-SmoothL1Loss
        training                 (min:   -6.244, max:   -0.067, cur:   -6.244)
        validation               (min:   -7.431, max:   -0.676, cur:   -7.428)
[21]:
metrics.tail(1)
[21]:
SmoothL1Loss MAE RMSE RegLoss SmoothL1Loss_val MAE_val RMSE_val
108 0.001941 29.853179 52.040566 0.0 0.000594 17.883106 30.412052
[22]:
m = m.highlight_nth_step_ahead_of_each_forecast(1)
fig_param = m.plot_parameters()
_images/energy_solar_pv_27_0.png

1-step ahead forecast with AR-Net: Using a Neural Network

There is something to consider here, if we consider a neural network with at least one hidden layer: Learning Rate matters when training a Neural Network.

For a high enough learning rate (probably > 0.1), the gradient seems to vanish and forces the AR net output to 0.
An easy way to void this issue is to set the learning rate at a low enough value, likely around 0.01 to 0.001.
[23]:
m = NeuralProphet(
    growth='off',
    yearly_seasonality=False,
    weekly_seasonality=False,
    daily_seasonality=False,
    n_lags=3*24,
    num_hidden_layers=4,
    d_hidden=8,
    learning_rate=0.003,
)

df = sf_pv_df.copy(deep=True)
df["I"] = np.append(0, sf_pv_df["y"].values[1:] - sf_pv_df["y"].values[:-1])
df_train, df_test = m.split_df(df, freq='H', valid_p = 0.10)
m = m.add_lagged_regressor("I", normalize="standardize")
metrics = m.fit(df_train, freq='H', validation_df=df_test, progress='plot')
_images/energy_solar_pv_29_0.png
log-SmoothL1Loss
        training                 (min:   -6.682, max:   -1.450, cur:   -6.682)
        validation               (min:   -7.519, max:   -1.754, cur:   -7.297)
[24]:
metrics.tail(1)
[24]:
SmoothL1Loss MAE RMSE RegLoss SmoothL1Loss_val MAE_val RMSE_val
108 0.001253 20.724888 41.326194 0.0 0.000677 15.56223 32.46497
[25]:
fig = m.plot_parameters()
_images/energy_solar_pv_31_0.png
[26]:
m = m.highlight_nth_step_ahead_of_each_forecast(1)
forecast = m.predict(df_test)
fig = m.plot(forecast)
_images/energy_solar_pv_32_0.png
[27]:
fig_comp = m.plot_components(forecast)
_images/energy_solar_pv_33_0.png
[28]:
fig_prediction = m.plot(forecast[-48:])
_images/energy_solar_pv_34_0.png
[ ]: