I want to forecast hourly temperature values using the fbprophet model. So far I have trained the model on just ds and y variable and it is giving me good results. But now I want to add extra regressors and then perform the forecast.
After fitting the model with extra regressors, I test it on test dataset which is giving me good accuracy. But the main issue is how to forecast future with it (data beyond my test set)
Here's what I've done so far.
# Data preparation and feature engineering
temp = df[["Temperature"]].apply(kelvinToDegC).copy()
temp["hourlyLag"] = temp["Temperature"].shift(1).bfill()
temp["dailyLag"] = temp["Temperature"].shift(24).bfill()
temp["weeklyLag"] = temp["Temperature"].shift(24*7).bfill()
temp["movMean"] = temp["Temperature"].rolling(window="24h").mean().bfill()
temp["mStd"] = temp["Temperature"].rolling(window="24h").std().bfill()
temp["ub"] = temp["movMean"] + (1.6 * temp["mStd"])
temp["lb"] = temp["movMean"] - (1.6 * temp["mStd"])
temp["devFromMean"] = temp["movMean"] - temp["Temperature"]
temp["devFromUB"] = temp["ub"] - temp["Temperature"]
temp["devFromLB"] = temp["lb"] - temp["Temperature"]
temp["hour"] = temp.index.hour
temp["dayOfYear"] = temp.index.day
temp["month"] = temp.index.month
temp = temp.reset_index()
temp.rename(columns={"Date": "ds", "Temperature": "y"}, inplace=True)
model = Prophet()
model.add_regressor("hourlyLag")
model.add_regressor("dailyLag")
model.add_regressor("weeklyLag")
model.add_regressor("movMean")
model.add_regressor("mStd")
model.add_regressor("ub")
model.add_regressor("lb")
model.add_regressor("devFromMean")
model.add_regressor("devFromUB")
model.add_regressor("devFromLB")
model.add_regressor("hour")
model.add_regressor("dayOfYear")
model.add_regressor("month")
model.fit(train)
These are the mae and mape scores
MAE: 0.00
MAPE: 0.17 %
now future = model.make_future_dataframe(periods=24 * 365 * 3, freq="h")
I am getting this error
ValueError Traceback (most recent call last)
Cell In[68], line 2
1 future = model.make_future_dataframe(periods=8760, freq="h")
----> 2 predictions = model.predict(future)
File c:\Users\5923imtiaz\AppData\Local\anaconda3\envs\ai\Lib\site-packages\prophet\forecaster.py:1270, in Prophet.predict(self, df, vectorized)
1268 if df.shape[0] == 0:
1269 raise ValueError('Dataframe has no rows.')
-> 1270 df = self.setup_dataframe(df.copy())
1272 df['trend'] = self.predict_trend(df)
1273 seasonal_components = self.predict_seasonal_components(df)
File c:\Users\5923imtiaz\AppData\Local\anaconda3\envs\ai\Lib\site-packages\prophet\forecaster.py:297, in Prophet.setup_dataframe(self, df, initialize_scales)
295 for name in self.extra_regressors:
296 if name not in df:
--> 297 raise ValueError(
298 'Regressor {name!r} missing from dataframe'
299 .format(name=name)
300 )
301 df[name] = pd.to_numeric(df[name])
302 if df[name].isnull().any():
ValueError: Regressor 'hourlyLag' missing from dataframe
I want to forecast hourly temperature values using the fbprophet model. So far I have trained the model on just ds and y variable and it is giving me good results. But now I want to add extra regressors and then perform the forecast.
After fitting the model with extra regressors, I test it on test dataset which is giving me good accuracy. But the main issue is how to forecast future with it (data beyond my test set)
Here's what I've done so far.
# Data preparation and feature engineering
temp = df[["Temperature"]].apply(kelvinToDegC).copy()
temp["hourlyLag"] = temp["Temperature"].shift(1).bfill()
temp["dailyLag"] = temp["Temperature"].shift(24).bfill()
temp["weeklyLag"] = temp["Temperature"].shift(24*7).bfill()
temp["movMean"] = temp["Temperature"].rolling(window="24h").mean().bfill()
temp["mStd"] = temp["Temperature"].rolling(window="24h").std().bfill()
temp["ub"] = temp["movMean"] + (1.6 * temp["mStd"])
temp["lb"] = temp["movMean"] - (1.6 * temp["mStd"])
temp["devFromMean"] = temp["movMean"] - temp["Temperature"]
temp["devFromUB"] = temp["ub"] - temp["Temperature"]
temp["devFromLB"] = temp["lb"] - temp["Temperature"]
temp["hour"] = temp.index.hour
temp["dayOfYear"] = temp.index.day
temp["month"] = temp.index.month
temp = temp.reset_index()
temp.rename(columns={"Date": "ds", "Temperature": "y"}, inplace=True)
model = Prophet()
model.add_regressor("hourlyLag")
model.add_regressor("dailyLag")
model.add_regressor("weeklyLag")
model.add_regressor("movMean")
model.add_regressor("mStd")
model.add_regressor("ub")
model.add_regressor("lb")
model.add_regressor("devFromMean")
model.add_regressor("devFromUB")
model.add_regressor("devFromLB")
model.add_regressor("hour")
model.add_regressor("dayOfYear")
model.add_regressor("month")
model.fit(train)
These are the mae and mape scores
MAE: 0.00
MAPE: 0.17 %
now future = model.make_future_dataframe(periods=24 * 365 * 3, freq="h")
I am getting this error
ValueError Traceback (most recent call last)
Cell In[68], line 2
1 future = model.make_future_dataframe(periods=8760, freq="h")
----> 2 predictions = model.predict(future)
File c:\Users\5923imtiaz\AppData\Local\anaconda3\envs\ai\Lib\site-packages\prophet\forecaster.py:1270, in Prophet.predict(self, df, vectorized)
1268 if df.shape[0] == 0:
1269 raise ValueError('Dataframe has no rows.')
-> 1270 df = self.setup_dataframe(df.copy())
1272 df['trend'] = self.predict_trend(df)
1273 seasonal_components = self.predict_seasonal_components(df)
File c:\Users\5923imtiaz\AppData\Local\anaconda3\envs\ai\Lib\site-packages\prophet\forecaster.py:297, in Prophet.setup_dataframe(self, df, initialize_scales)
295 for name in self.extra_regressors:
296 if name not in df:
--> 297 raise ValueError(
298 'Regressor {name!r} missing from dataframe'
299 .format(name=name)
300 )
301 df[name] = pd.to_numeric(df[name])
302 if df[name].isnull().any():
ValueError: Regressor 'hourlyLag' missing from dataframe
1 Answer
Reset to default 0Here you are fitting 'train' instead of 'temp' which contains the extra regressors. Also note that you need future values for any extra regressor you pass to prophet.
When you create your future dataframe,
future = model.make_future_dataframe(periods=24 * 365 * 3, freq="h")
This expects to produce a 3 year forecast at hourly frequency resulting in > 20,000 rows. However the 'hourlyLag' regressor is only shifted down 1 position, assuming your 'Temperature' data ends at the start of your forecast horizon, this is not sufficient data to predict your forecast horizon length.
You can review the Prophet documentation for a more detailed explaination.