How to model short burst of arrivals with long gaps

Questions
1

Hi
I have a dataset with burst of arrivals spread between gaps with few arrivals. Any suggestion on how to model this using pymc3 ?
The bursts are similar and single exponential rate may be suitable and for the long gaps uniform rate may be suitable. But how to model multiple switch points ?

Thanks

2

I’d like to help you get started on this but more information would be helpful. Perhaps you could start by indicating relevant papers or blog posts that have models similar to what you’re interested in. Could you clarify the type and/or support of the data you have? Is this count data?

3

Thank you.

The data is event count data aggregated over 15 min intervals.

data_plot
data_plot1158Ă—299 20.9 KB

I am attaching the data points as csvevents.csv (16.4 KB)

I have not been able to find any example or posts dealing with state and associated rate with the state.

Any suggestions on how to begin will be great.

4

I don’t think there is enough information here to advise. I don’t understand the motivation or the question you are trying to answer.

5

Hi
The scenario is like this: When the system is in state-A (let’s say normal state) the number of events is zero and when one event occurs the system moves to state-B (problem state) and there is a burst of events for a period of time after which the system returns to state-A.
I am looking to model the duration of state-A and duration of state-B and the intensity of events in state-A (zero to 1) and state-B.

Thanks

6

I think I am starting to understand what kind of model you might want. I have another question for you - does your data involve transition from state A to state B every time? Also, can it transition from B back to A?

7

Yes, the system transitions from State A to State B every time and I am thinking that the time duration between the states may be something which could be modeled as a normal distribution.

Another way, will be to model two poisson processes, one each for state A and state B, and get the rates of those distributions. And system probability distribution of being is either state A or state B is generated as joint distribution. I think this may be a better approach and the time duration between states may not be close to normal.

Yes, the system transitions backs from B to A.

Another example of the system events counts

StateA-StateB-ID5
StateA-StateB-ID5927Ă—299 43.3 KB

Thanks

8

Hi @ckrapu Any suggestions on how to approach this with pymc3.

Thanks

9

Hi @praveen049

The fact that you’re talking about two stats; with measured outputs being different in the two states, immediately says that you probably want to model this as a Markov chain. What you have looks to be very nearly identical to a model considered in this blog post:

1 Like