Simulation of a homogeneous Markov chain (in continuous time)
Markov chains in continuous time are described by their Q-matrix, this is also referred to as generator or rate matrix. Assuming that there are only finitely many states, and neither of them is absorbent, i.e. that each state can be exited again, applies to this matrix: The entries outside the main diagonal are non-negative, the entries on the main diagonal are negative, have the row sums to 0.
Especially for the simulation the following interpretation of the entries is important: the residence time in a condition is exponentially distributed, the parameter is the amount of the associated main diagonal element. After the dwell time in the state i, the process jumps to a new state, the probability of a transition to j can be calculated by the rate qij divied by |qii|. The probability for a jump in the same state is 0, because the residence time has been expired.
If the residence times are ignored, i.e. only the jumps are considered, there is a Markov chain in discrete time, the embedded jump chain. Therefore, can a Markov chain in continuous time with a finite number of non-absorbing states can be composed from a Markov chain in discrete time and a series of exponentially distributed residence times.
Transition rates: They determine how the Markov chain to be simulated looks like: The number of states and the transition rates has to be entered. Below this the transition matrix of the embedded jump chain is automatically calculated.
State function: The states of the Markov chain will be displayed here. The relative frequencies for each state can be read from the histogram below.
Markov graph: The graph is displayed, the numbers on the edges give the transition probability; the numbers at the nodes are the parameters of the exponential distributed residence times . The current state in the current simulation step is highlighted, as well as the last-used edge. The relative frequency of the states are represented by the brightness levels.