I’m trying to use state space’s forecast method with exogenous variables, but I keep getting this error:
ValueError: conflicting sizes for dimension ‘time’: length 1107 on the data but length 8 on coordinate ‘time’
My scenario: I’m passing exogenous indicator variables totaling 18 columns and 8 rows long, for an 8 day forecast. The period I’m forecasting is not at the -very- end of my time series. I’m testing my exogenous variable’s ability to reduce variance on events and holidays.
Time series info:
Length: 1107 data points
Start: 2022-01-19
End: 2025-01-29
Relevant code:
# Define start date and forecast period
start_date, n_periods = pd.to_datetime("2024-1-25"), 8
# Extract exogenous indicator variables for the forecast period
scenario = {'data_exog': pd.DataFrame(df_holidays.loc[start_date:].iloc[:n_periods].to_numpy(dtype=float),
columns=df_holidays.columns)}
# Generate the forecast
forecasts = ss_mod.forecast(trace, start=start_date, periods=n_periods, scenario=scenario)
My second line of code is extracting the relevant portion of my exogenous variables and assigning it to the scenario.
Am I getting this error because I’m not forecasting at the very end of my time series? (edit: dumb question, I’ll test this later.)
Hey @rOy_bOy, I have come across this issue before. I believe that at the moment you need to employ a hack by deleting the dims in your state space model object like:
del ssm_model._exog_data_info[exogenous_data_name]["dims"]
Sounds like a bug, can you open an issue on the extras repo and I’ll try to identify exactly what needs to be done? I thought I was testing this case, so I’m a bit miffed it’s still giving you guys problems.
Gave that a shot. I was able to complete the forecast inference, but plotting the results showed no data? I haven’t tried getting into the guts of the output yet.
Hey @rOy_bOy, could you share the model code you are using? If you can also share a working example that would be great but I understand if your data is confidential. I’d be happy to help you debug this issue.
Couple caveats: 1) I’ve got some nasty shape on the posterior, I think, so usually one of my chains hangs for a bit. 2) Sorry for the code bloat - I was using some AI (aren’t we all?) to fast iterate.
Thanks for opening an issue. If you can make a smaller example that reproduces that problem (that I can copy/paste and run locally) it would be extremely helpful. You can just use generated data.
Hey @rOy_bOy, I was able to run through your code and get forecasts that are NOT NaNs. I noticed that you are using the StateSpace Module from the old pymc-experimentals package. I recommend you move over to using StateSpace from pymc-extras. Here is a gist of your code that I ran through. I just commented out the plots. Please let me know if you still have troubles with this.
time (time) datetime64[ns] 64B 2024-04-16 … 2024-04-23
state (state) <U27 3kB ‘level’ … ‘New_Years_Day_post_sq’
observed_state (observed_state) <U4 16B ‘data’
Data variables:
forecast_latent (chain, draw, time, state) float64 6MB nan nan … nan
forecast_observed (chain, draw, time, observed_state) float64 192kB nan …
Attributes:
created_at: 2025-02-12T21:59:02.212224+00:00
arviz_version: 0.20.0
inference_library: pymc
inference_library_version: 5.20.1
As you can see, the output has NaN’s. The only change I made to the model was to include the newly named state space module and remove the reference to the old library.
from pymc_extras.statespace import structural as st
Was that the only change you made to my model in order to get your output? I looked thru the code, but I didn’t see any other changes. Also, how did you get my model to run in 41 minutes O_o ??
I’m gonna reset my kernel and try running this again.
Hey @rOy_bOy, can you try creating a clean environment before running. Something may be messed up with your environment. Also, I don’t believe that I changed anything with your model. I had to make a couple adjustments to the data part because the data you uploaded didn’t have column names. You could just download the code I uploaded from the gist above and run it. That is exactly the code that I ran on my end. I don’t really know if 41 minutes is fast or slow for this model but I am running on an MacBook laptop that has an m4 pro chip.
Brutal. My buddy with a Macbook Pro yesterday was teasing me that he thought his computer would out perform my computer with this model. I told him that’s not possible.
This is terrible news.
However, yeah, I’m gonna build a new environment and proceed.
With nutpie you can get away with fewer tuning steps, maybe try 500/500.
I’ve also been using pymc_extras.fit_laplace lately to iterate on models more quickly, that should fit in just a couple minutes max.
I will say that I just got an m4 max and my experience fitting models has been a significant speedup, but I’ve also had to play with BLAS thread settings
@jessegrabowski Thank you for that resource! I will have to check it out and see the performance I gain from tweaking those settings. I will also have to try out the fit_laplace method for fast iterations. Are there any caveats/gotcha’s to look out for when using fit_laplace?
Make sure you set compile_kwargs = {'mode': 'JAX'} and gradient_backend='jax'.
After that, for solvers, pick one that uses hessian for faster convergence ("Newton-CG" is a good default, or "trust-ncg" for trickier problems). Set use_jac=True, use_hessp=True (use_hess will compute the entire hessian which is not necessary, use_hessp will only compute the hessian-vector product, so it’s orders of magnitude faster with no performance penalty)
Alright, I have good news of great rejoicing. After creating several new environments, because I couldn’t get the silly BLAS connections right on pytensor, I finally got an environment that was able to run the model (in 4 hours >:-/ ). Then I used the hack and I got results (no NaN’s)! I should have tried it first without the hack. But I’ll redo it again later.