Test and CrossValidate#
[1]:
if "google.colab" in str(get_ipython()):
# uninstall preinstalled packages from Colab to avoid conflicts
!pip uninstall -y torch notebook notebook_shim tensorflow tensorflow-datasets prophet torchaudio torchdata torchtext torchvision
#!pip install git+https://github.com/ourownstory/neural_prophet.git # may take a while
!pip install neuralprophet # much faster, but may not have the latest upgrades/bugfixes
import pandas as pd
from neuralprophet import NeuralProphet, set_log_level
set_log_level("ERROR")
Load data#
[2]:
data_location = "https://raw.githubusercontent.com/ourownstory/neuralprophet-data/main/datasets/"
df = pd.read_csv(data_location + "air_passengers.csv")
1. Basic: Train and Test a model#
First, we show how to fit a model and evaluate it on a holdout set.
1.1 Train-Test evaluation#
[3]:
m = NeuralProphet(seasonality_mode="multiplicative", learning_rate=0.1)
m.set_plotting_backend("plotly-static")
df = pd.read_csv(data_location + "air_passengers.csv")
df_train, df_test = m.split_df(df=df, freq="MS", valid_p=0.2)
metrics_train = m.fit(df=df_train, freq="MS")
metrics_test = m.test(df=df_test)
metrics_test
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.002764572622254491
MAE_val 18.907012939453125
RMSE_val 23.143999099731445
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[3]:
| MAE_val | RMSE_val | Loss_test | RegLoss_test | |
|---|---|---|---|---|
| 0 | 18.907013 | 23.143999 | 0.002765 | 0.0 |
1.2 Predict into future#
Before making any actual forecasts, re-fit the model on all data available, else you are greatly reducing your forecast accuracy!
[4]:
m = NeuralProphet(seasonality_mode="multiplicative", learning_rate=0.1)
m.set_plotting_backend("plotly-static")
metrics_train2 = m.fit(df=df, freq="MS")
future = m.make_future_dataframe(df, periods=24, n_historic_predictions=48)
forecast = m.predict(future)
m.plot(forecast)
1.3 Visualize training#
If you installed the [live] version of NeuralProphet, you can additionally visualize your training progress and spot any overfitting by evaluating every epoch.
Note: Again, before making any predictions, re-fit the model with the entire data first.
[5]:
m = NeuralProphet(seasonality_mode="multiplicative", learning_rate=0.1)
m.set_plotting_backend("plotly-static")
df = pd.read_csv(data_location + "air_passengers.csv")
df_train, df_test = m.split_df(df=df, freq="MS", valid_p=0.2)
metrics = m.fit(df=df_train, freq="MS", validation_df=df_test, progress="plot")
[6]:
metrics.tail(1)
[6]:
| MAE_val | RMSE_val | Loss_val | RegLoss_val | epoch | MAE | RMSE | Loss | RegLoss | |
|---|---|---|---|---|---|---|---|---|---|
| 491 | 19.501102 | 23.570879 | 0.002867 | 0.0 | 491 | 6.099627 | 7.32115 | 0.000215 | 0.0 |
2. Time-series Cross-Validation#
Time-series cross-validation is a technique that is also referred to as a rolling origin backtest. It involves dividing the data into several folds. * During the first fold, we train the model on a portion of the data and then evaluate its performance on the next set of data points, which are determined by the fold_pct parameter (percentage of samples in each fold). * In the next fold, we include the evaluation data from the previous fold in the training data and then evaluate the model’s performance on a later set of data points. * This process is repeated until the final fold, where the evaluation data reaches the end of the available data. Essentially, the forecast origin “rolls” forward as we move from one fold to the next.
Note: Before making any actual forecasts, re-fit the model on all data available, else you are greatly reducing your forecast accuracy!
[7]:
METRICS = ["MAE", "RMSE"]
METRICS_VAL = ["MAE_val", "RMSE_val"]
params = {"seasonality_mode": "multiplicative", "learning_rate": 0.1}
df = pd.read_csv(data_location + "air_passengers.csv")
folds = NeuralProphet(**params).crossvalidation_split_df(df, freq="MS", k=5, fold_pct=0.20, fold_overlap_pct=0.5)
[8]:
metrics_train = pd.DataFrame(columns=METRICS)
metrics_test = pd.DataFrame(columns=METRICS_VAL)
for df_train, df_test in folds:
m = NeuralProphet(**params)
m.set_plotting_backend("plotly-static")
train = m.fit(df=df_train, freq="MS")
test = m.test(df=df_test)
metrics_train = metrics_train.append(train[METRICS].iloc[-1])
metrics_test = metrics_test.append(test[METRICS_VAL].iloc[-1])
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/138744510.py:9: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/138744510.py:10: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.01086195558309555
MAE_val 16.587053298950195
RMSE_val 20.34723472595215
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/138744510.py:9: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/138744510.py:10: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.02300422266125679
MAE_val 31.630748748779297
RMSE_val 34.3193244934082
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/138744510.py:9: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/138744510.py:10: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.009417595341801643
MAE_val 21.363872528076172
RMSE_val 28.63540267944336
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/138744510.py:9: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/138744510.py:10: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.0073935664258897305
MAE_val 26.357913970947266
RMSE_val 30.63770866394043
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/138744510.py:9: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/138744510.py:10: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.0026712114922702312
MAE_val 18.709611892700195
RMSE_val 22.74985122680664
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[9]:
metrics_test.describe().loc[["mean", "std", "min", "max"]]
[9]:
| MAE_val | RMSE_val | |
|---|---|---|
| mean | 22.929840 | 27.337904 |
| std | 6.081756 | 5.727830 |
| min | 16.587053 | 20.347235 |
| max | 31.630749 | 34.319324 |
2. Advanced: 3-Phase Train, Validate and Test procedure#
Finally, in 2.1 and 2.2, we will do a 3-part data split to do a proper training, validation and test evaluation of your model. This setup is used if you do not want to bias your performance evaluation by your manual hyperparameter tuning. this is, however not common when working with time series, unless you work in academia. Crossvalidation is usually more than adequate to evaluate your model performance.
If you are confused by this, simply ignore this section and continue your forecasting life. Or if you got curious, read up on how to evaluate machine learning models to level up your skills.
2.1 Train, Validate and Test evaluation#
[10]:
m = NeuralProphet(seasonality_mode="multiplicative", learning_rate=0.1)
m.set_plotting_backend("plotly-static")
df = pd.read_csv(data_location + "air_passengers.csv")
# create a test holdout set:
df_train_val, df_test = m.split_df(df=df, freq="MS", valid_p=0.2)
# create a validation holdout set:
df_train, df_val = m.split_df(df=df_train_val, freq="MS", valid_p=0.2)
# fit a model on training data and evaluate on validation set.
metrics_train1 = m.fit(df=df_train, freq="MS")
metrics_val = m.test(df=df_val)
# refit model on training and validation data and evaluate on test set.
m = NeuralProphet(seasonality_mode="multiplicative", learning_rate=0.1)
m.set_plotting_backend("plotly-static")
metrics_train2 = m.fit(df=df_train_val, freq="MS")
metrics_test = m.test(df=df_test)
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.005187277216464281
MAE_val 18.062246322631836
RMSE_val 25.076841354370117
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.0026784376241266727
MAE_val 18.72081184387207
RMSE_val 22.78059959411621
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[11]:
metrics_train1["split"] = "train1"
metrics_train2["split"] = "train2"
metrics_val["split"] = "validate"
metrics_test["split"] = "test"
metrics_train1.tail(1).append([metrics_train2.tail(1), metrics_val, metrics_test]).drop(columns=["RegLoss"])
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/302924761.py:5: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
[11]:
| MAE | RMSE | Loss | epoch | split | MAE_val | RMSE_val | Loss_test | RegLoss_test | |
|---|---|---|---|---|---|---|---|---|---|
| 563 | 5.339781 | 6.604742 | 0.000249 | 563.0 | train1 | NaN | NaN | NaN | NaN |
| 491 | 6.298254 | 7.553654 | 0.000226 | 491.0 | train2 | NaN | NaN | NaN | NaN |
| 0 | NaN | NaN | NaN | NaN | validate | 18.062246 | 25.076841 | 0.005187 | 0.0 |
| 0 | NaN | NaN | NaN | NaN | test | 18.720812 | 22.780600 | 0.002678 | 0.0 |
2.2 Train, Cross-Validate and Cross-Test evaluation#
[12]:
METRICS = ["MAE", "RMSE"]
METRICS_VAL = ["MAE_val", "RMSE_val"]
params = {"seasonality_mode": "multiplicative", "learning_rate": 0.1}
df = pd.read_csv(data_location + "air_passengers.csv")
folds_val, folds_test = NeuralProphet(**params).double_crossvalidation_split_df(
df, freq="MS", k=5, valid_pct=0.10, test_pct=0.10
)
[13]:
metrics_train1 = pd.DataFrame(columns=METRICS)
metrics_val = pd.DataFrame(columns=METRICS_VAL)
for df_train1, df_val in folds_val:
m = NeuralProphet(**params)
m.set_plotting_backend("plotly-static")
train1 = m.fit(df=df_train, freq="MS")
val = m.test(df=df_val)
metrics_train1 = metrics_train1.append(train1[METRICS].iloc[-1])
metrics_val = metrics_val.append(val[METRICS_VAL].iloc[-1])
metrics_train2 = pd.DataFrame(columns=METRICS)
metrics_test = pd.DataFrame(columns=METRICS_VAL)
for df_train2, df_test in folds_test:
m = NeuralProphet(**params)
m.set_plotting_backend("plotly-static")
train2 = m.fit(df=df_train2, freq="MS")
test = m.test(df=df_test)
metrics_train2 = metrics_train2.append(train2[METRICS].iloc[-1])
metrics_test = metrics_test.append(test[METRICS_VAL].iloc[-1])
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:8: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:9: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.017169639468193054
MAE_val 43.81590270996094
RMSE_val 45.62299346923828
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:8: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:9: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.01694938912987709
MAE_val 42.67584228515625
RMSE_val 45.32938766479492
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:8: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:9: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.008688823319971561
MAE_val 29.468582153320312
RMSE_val 32.455142974853516
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:8: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:9: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.001547039020806551
MAE_val 13.694732666015625
RMSE_val 13.694735527038574
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:8: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:9: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.0036992093082517385
MAE_val 21.072723388671875
RMSE_val 21.17667007446289
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:18: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:19: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.0032182412687689066
MAE_val 28.705841064453125
RMSE_val 28.857887268066406
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:18: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:19: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.0016795611009001732
MAE_val 17.63250732421875
RMSE_val 20.83587646484375
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:18: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:19: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.0010085979010909796
MAE_val 12.40789794921875
RMSE_val 16.33718490600586
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:18: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:19: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.0016017681919038296
MAE_val 18.285919189453125
RMSE_val 20.88248634338379
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:18: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
WARNING - (py.warnings._showwarnmsg) - /var/folders/6b/n_b96k8n2pn66yjx0387dhjc0000gn/T/ipykernel_22660/2088809072.py:19: FutureWarning:
The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Loss_test 0.0005868254229426384
MAE_val 11.258453369140625
RMSE_val 13.225532531738281
RegLoss_test 0.0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[14]:
metrics_train2.describe().loc[["mean", "std"]]
[14]:
| MAE | RMSE | |
|---|---|---|
| mean | 7.470476 | 9.342587 |
| std | 0.249411 | 0.270676 |
[15]:
metrics_val.describe().loc[["mean", "std"]]
[15]:
| MAE_val | RMSE_val | |
|---|---|---|
| mean | 30.145557 | 31.655786 |
| std | 13.203131 | 14.274982 |
[16]:
metrics_test.describe().loc[["mean", "std"]]
[16]:
| MAE_val | RMSE_val | |
|---|---|---|
| mean | 17.658124 | 20.027794 |
| std | 6.909549 | 5.900114 |