Stochastic Processs
Published:
Possion Process
The stochastic countng satifies the following conditions is defined as Possion Process
- $N(0) = 0$
- $N(t)$ is indepentent increment process
\((X_{t_1}-_{t_0}),\dots,(X_{t_{n+1}}-_{t_n})\) are indepentent
- $N(t)$ is statoinary increment process
Using Generating function $G(z,t)=E(z^(N(t)))=sum_k z^k P(N(t)=k)$ to find that possion process is
\[P(N(t) = k) = \frac{(\lambda t)^k}{k!} \exp{-\lambda t}\]Markov Process
\[P(X_n=x_n|X_{n-1}=x_{n-1},\dots,X_{1}=x_1)=P(X_n=x_n|X_{n-1}=x_{n-1})\]Chapman–Kolmogorov equation
\[P_{i,j}(n)=\sum_k P_{i,k}(n) P_{k,j}(n)\]Reachable
- when $i,j$ is reachable, there exists $n$ such that $P_{i,j}(n)>0$, i.e. $i \rightarrow j$
Commutative
- $i \leftrightarrow j$ that means $i \rightarrow j$ and $j \rightarrow i$
Closed
- a set $S$ is defined as closed if a subset $C$ of $S$, $a \in C$, $b \notin C$ , which imples $a \not\to b$
Irreducible
- a set $S$ is irreducible if $S$ do not include closed true subset $C$
- iff $S$ is closed and all elements in S is commutative
First passaege Proability
\[f_{i,j}(n)=P(X_n=j,X_{n-1}\not= j,\dots, X_1\not=j|X_{n-1}=i)\] \[0\le \sum_{n} f_{i,j}(n) \le 1\]Recurrent
\(\sum_{n} f_{i,j}(n) =1\)
or
\[\sum_{n} P_{i,j}(n)=\infty\]- the proof is using
where
\[P_{i,j}(z)=\sum_{n}P_{i,j}(n)z^n\]- finite state has an elemnent which is recurrent
- irreducibe with finite state implies all states are recurrent
Periodic
- irreducible and recurrent, the following holds.
- positive recurrent:
- null recurrent:
- when $d_i=1$, the diagram is non-periodic
stable converage
if finite state Diagram D is irreducible and non-perodic, then \(P_{i,j}(n) \rightarrow \pi_{i,j}\)
if finite states Diagram D is irreducibe and recurrent, then