Cross Validation with regressors

[ghost]

I am having issues using the cross_validation function when I add regressors to my prophet function. I have not been able to find any documentation within prophet nor have I seen any examples of people using cross_validation() after adding regressors to their model. The error I receive is always

"Error in setup_dataframe(object, df) :
Regressor "bookings" missing from dataframe"

[roumail]

Hey,

Just a quick check: When you add regressors, you need to add them to the training data and then specify them using add_regressors. The latter function seems to fail because the regressor bookings isn't found in the training data you pass to cross_validation.

Are you specifying the same name?

[ghost]

So I add the regressors to my model using add_regressor and then I try to run cross validation on that same model similar to below. I was hoping the model would save the necessary information for cross validation to be run with the regressors in mind. Does a similar approach with add_regressors() need to be used with the cross_validation function? m2 <- prophet(daily.seasonality = FALSE, yearly.seasonality = TRUE) m2 <- add_regressor(m2, "bookings") m2 <- fit.prophet(m2,prophet_df) cv2 <- cross_validation(m2, horizon = 90, units = "days")

…

On Fri, Dec 15, 2017 at 3:45 AM, roumail ***@***.***> wrote: Hey, Just a quick check: When you add regressors, you need to add them to the training data and then specify them using add_regressors. The latter function seems to fail because the regressor bookings isn't found in the training data you pass to cross_validation. Are you specifying the same name? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#380 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHIXcm5b0ASu97rKFz_JauDqweAHTh0eks5tAj_VgaJpZM4RBnFN> .

[roumail]

Does prophet_df contain a column by the name bookings?

[ghost]

Yes. The fit.prophet function trains the model correctly with the specified regressor present. For cross_validation I provide the same model and it fails saying the regressor is not present.

…

On Dec 18, 2017 2:44 PM, "roumail" ***@***.***> wrote: Does prophet_df contain a column by the name bookings? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#380 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHIXcvF4JgJZgWfstfFA3gZaHRKFExRdks5tBs7FgaJpZM4RBnFN> .

[bletham]

This is a bug, sorry about that. See also #351.

It has been fixed in the current github version, which you can install with

devtools::install_github('facebook/prophet', subdir='R', ref='master')

We'll be pushing the fix to CRAN soon.

[rpanai]

@bletham is there a similar bug on the python version too?

[bletham]

@rpanai yes this bug exists in the python version also. It is fixed fixed in the current master branch here on github, so if you download and install that it will work correctly.

[nielsenmarkus11]

@bletham, I have a quick clarification question. When prophet cross_validates in the rolling window with regressors that are also time series (eg. using temperature to predict pollution), does it actually predict the regressors for the horizon as well as the response variable? Or does it use the actual data from the regressors? Thanks,

[bletham]

It does not predict the regressors. They are taken as "known" and so the true values are used for them. If the future values of the regressors are in reality coming from another Prophet forecast then it would be correct (and probably important) to also forecast the regressors like you described.

Currently we don't know where future values for regressors are coming from (deterministic? prophet? some other forecasting method?) and so there wouldn't be a way to do this. But making an interface to cross_validate that makes this possible would be good, I made an issue for this in #442.

By the way, if you have a successful example of using other prophet forecasts as extra regressors, I'd love to see it.

[nielsenmarkus11]

Thanks @bletham . I currently just have my code in a simple for loop running prophet() for each of the regressors then I do the same for the response variable using the yhat values from each of the forecasted regressors. I'll take a look at the issue you provided above and see if I can send a simple example to you. I'm assuming you'd prefer it all to be automated within the cross_validate function?

[bletham]

That seems reasonable.

Maybe for starters we'd keep it as a separate function, for cross validation with Prophet extra regressors. I think we'd need to pass in the prophet models for each of the regressors to have correct parameter settings for them in the CV.

[nielsenmarkus11]

You can find my loop here in one of my repositories. I'm using temperature, wind, and precipitation to predict PM2.5. Let me know how else I can help.

[bletham]

I'm going to close this to merge it with #351, which is tracking the root issue.