Gamma Distribution

A general type of statistical Distribution which is related to the Beta Distribution and arises naturally in processes for which the waiting times between Poisson Distributed events are relevant. Gamma distributions have two free parameters, labeled $\alpha$ and $\theta$, a few of which are illustrated above.

Given a Poisson Distribution with a rate of change $\lambda$, the Distribution Function $D(x)$ giving the waiting times until the $h$th change is

$$D(x) = P(X \leq x) = 1-P(X > x)$$

$$= 1 - \sum_{k=0}^{h-1} {(\lambda x)^ke^{-\lambda x}\over k!}$$

$$= 1 - e^{-\lambda x} \sum_{k=0}^{h-1} { (\lambda x)^k\over k!}$$ (1)

for $x\geq 0$. The probability function $P(x)$ is then obtained by differentiating $D(x)$,

$$P(x) = D'(x)$$

$$= \lambda e^{-\lambda x} \sum_{k=0}^{h-1} { (\lambda x)^k\over k!} - e^{-\lambda x} \sum_{k=0}^{h-1} { k(\lambda x)^{k-1}\lambda\over k!}$$

$$= \lambda e^{-\lambda x} + \lambda e^{-\lambda x} \sum_{k=1}^{h-1} ... ...ver k!} - e^{-\lambda x} \sum_{k=1}^{h-1} { k(\lambda x)^{k-1}\lambda \over k!}$$

$$= \lambda e^{-\lambda x} - \lambda e^{-\lambda x} \sum_{k=1}^{h-1} \left[{{ k(\lambda x)^{k-1}\over k!} - { (\lambda x)^k\over k!}}\right]$$

$$= \lambda e^{-\lambda x}\left\{{ 1 - \sum_{k=1}^{h-1} \left[{ (\lambda x)^{k-1}\over (k-1)!} - { (\lambda x)^k\over k!}\right]}\right\}$$

$$= \lambda e^{-\lambda x}\left\{{ 1 - \left[{1 - { (\lambda x)^{h-1}... ...)!}}\right]}\right\} = { \lambda (\lambda x)^{h-1}\over (h-1)!} e^{-\lambda x}.$$ (2)

Now let $\alpha \equiv h$ and define $\theta \equiv 1/\lambda$ to be the time between changes. Then the above equation can be written

$$P(x) = \cases{ {x^{\alpha-1}e^{-x/\theta}\over \Gamma (\alpha)\theta^\alpha} & $0 \leq x < \infty$\cr 0 & $x < 0$.\cr}$$ (3)

The Characteristic Function describing this distribution is

$$\phi(t)=(1-it)^{-p},$$ (4)

and the Moment-Generating Function is

$$M(t) = \int_0^\infty {e^{tx}x^{\alpha-1}e^{-x/\theta}\,dx\over \Gamma (\alpha)\theta^\alpha}$$

$$= \int_0^\infty {x^{\alpha-1}e^{-(1-\theta t)x/\theta}\,dx\over \Gamma (\alpha)\theta^\alpha}.$$ (5)

In order to find the Moments of the distribution, let

$$y \equiv {(1-\theta t)x\over\theta}$$ (6)

$$dy = {1-\theta t\over\theta}\,dx,$$ (7)

so

$$M(t) = \int_0^\infty\left({ \theta y\over 1-\theta t}\right)^{\alpha-1} {e^{-y}\over \Gamma (\alpha)\theta^\alpha} { \theta\,dy\over 1-\theta t}$$

$$= {1\over(1-\theta t)^\alpha \Gamma(\alpha)} \int^\infty_0 y^{\alpha-1}e^{-y}\,dy$$

$$= {1\over(1-\theta t)^\alpha},$$ (8)

and the logarithmic Moment-Generating function is

$$R(t) \equiv \ln M(t) = -\alpha \ln(1-\theta t)$$ (9)

$$R'(t) = { \alpha\theta\over 1-\theta t}$$ (10)

$$R''(t) = { \alpha\theta^2\over (1-\theta t)^2}.$$ (11)

The Mean, Variance, Skewness, and Kurtosis are then

$$\mu = R'(0) = \alpha\theta$$ (12)

$$\sigma^2 = R''(0) = \alpha\theta^2$$ (13)

$$\gamma_1 = {2\over\sqrt{\alpha}}$$ (14)

$$\gamma_2 = {6\over\alpha}.$$ (15)

The gamma distribution is closely related to other statistical distributions. If $X_1$, $X_2$, ..., $X_n$ are independent random variates with a gamma distribution having parameters $(\alpha_1, \theta)$, $(\alpha_2, \theta)$, ..., $(\alpha_n, \theta)$, then $\sum_{i=1}^n X_i$ is distributed as gamma with parameters

$$\alpha = \sum_{i=1}^n\alpha_i$$ (16)

$$\theta = \theta.$$ (17)

Also, if $X_1$ and $X_2$ are independent random variates with a gamma distribution having parameters $(\alpha_1, \theta)$ and $(\alpha_2, \theta)$, then $X_1/(X_1+X_2)$ is a Beta Distribution variate with parameters $(\alpha_1, \alpha_2)$. Both can be derived as follows.

$$P(x,y)={1\over\Gamma(\alpha_1)\Gamma(\alpha_2)} e^{x_1+x_2}{x_1}^{\alpha_1-1}{x_2}^{\alpha_2-1}.$$ (18)

Let

$$u=x_1+x_2 \qquad x_1=uv$$ (19)

$$v={x_1\over x_1+x_2} \qquad x_2=u(1-v),$$ (20)

then the Jacobian is

$$J\left({x_1,x_2\over u,v}\right)= \left\vert\matrix{v & u\cr 1-v & -u\cr}\right\vert = -u,$$ (21)

so

$$g(u,v)\,du\,dv=f(x,y)\,dx\,dy=f(x,y)u\,du\,dv.$$ (22)

$$g(u,v) = {u\over\Gamma(\alpha_1)\Gamma(\alpha_2)}e^{-u}(uv)^{\alpha_1-1}u^{\alpha_2-1}(1-v)^{\alpha_2-1}$$

$$= {1\over\Gamma(\alpha_1)\Gamma(\alpha_2)} e^{-u}u^{\alpha_1+\alpha_2-1}v^{\alpha_1-1}(1-v)^{\alpha_2-1}.$$ (23)

The sum $X_1+X_2$ therefore has the distribution

$$f(u)=f(x_1+x_2)=\int_0^1 g(u,v)\,dv={e^{-u}u^{\alpha_1+\alpha_2-1}\over \Gamma(\alpha_1+\alpha_2)},$$ (24)

which is a gamma distribution, and the ratio $X_1/(X_1+X_2)$ has the distribution

$$h(v) = h\left({x_1\over x_1+x_2}\right)=\int_0^\infty g(u,v)\,du$$

$$= {v^{\alpha_1-1}(1-v)^{\alpha_2-1}\over B(\alpha_1, \alpha_2)},$$ (25)

where $B$ is the Beta Function, which is a Beta Distribution.

If $X$ and $Y$ are gamma variates with parameters $\alpha_1$ and $\alpha_2$, the $X/Y$ is a variate with a Beta Prime Distribution with parameters $\alpha_1$ and $\alpha_2$. Let

$$u=x+y\qquad v={x\over y},$$ (26)

then the Jacobian is

$$J\left({u,v\over x,y}\right)= \left\vert\matrix{1 & 1\cr {1\... ...over y^2}\cr}\right\vert = -{x+y\over y^2} =-{(1+v)^2\over u},$$ (27)

so

$$dx\,dy={u\over(1+v)^2}\,du\,dv$$ (28)

$$g(u,v) = {1\over\Gamma(\alpha_1)\Gamma(\alpha_2)}e^{-u}\left({uv\over 1+v}\right)^{\alpha_1-1}\left({u\over 1+v}\right)^{\alpha_2-1} {u\over(1+v)^2}$$

$$= {1\over\Gamma(\alpha_1)\Gamma(\alpha_2)} e^{-u}u^{\alpha_1+\alpha_2-1}v^{\alpha_2-1}(1+v)^{-\alpha_1-\alpha_2}.$$ (29)

The ratio $X/Y$ therefore has the distribution

$$h(v)=\int_0^\infty g(u,v)\,du={v^{\alpha_1-1}(1+v)^{-\alpha_1-\alpha_2}\over B(\alpha_1,\alpha_2)},$$ (30)

which is a Beta Prime Distribution with parameters $(\alpha_1, \alpha_2)$.

The ``standard form'' of the gamma distribution is given by letting $y\equiv x/\theta$, so $dy=dx/\theta$ and

$$P(y)\,dy = {x^{\alpha-1}e^{-x/\theta}\over \Gamma(\alpha)\theta^\alpha}\,dx = {(\theta y)^{\alpha-1}e^{-y}\over \Gamma(\alpha)\theta^\alpha} \,(\theta\,dy)$$

$$= {y^{\alpha-1}e^{-y}\over\Gamma(\alpha)}\,dy,$$ (31)

so the Moments about 0 are

$$\nu_r = {1\over \Gamma(\alpha)} \int_0^\infty e^{-x}x^{\alph... ...r}\,dx = {\Gamma(\alpha+r)\over \Gamma(\alpha)} = (\alpha)_r,$$ (32)

where $(\alpha)_r$ is the Pochhammer Symbol. The Moments about $\mu=\mu_1$ are then

$$\mu_1 = \alpha$$ (33)

$$\mu_2 = \alpha$$ (34)

$$\mu_3 = 2\alpha$$ (35)

$$\mu_4 = 3\alpha^2+6\alpha.$$ (36)

The Moment-Generating Function is

$$M(t)={1\over(1-t)^\alpha},$$ (37)

and the Cumulant-Generating Function is

$$K(t)=\alpha\ln(1-t) = \alpha\left({t+{\textstyle{1\over 2}}t^2+{\textstyle{1\over 3}} t^3+\ldots}\right),$$ (38)

so the Cumulants are

$$\kappa_r = \alpha \Gamma(r).$$ (39)

If $x$ is a Normal variate with Mean $\mu$ and Standard Deviation $\sigma$, then

$$y\equiv {(x-\mu)^2\over 2\sigma^2}$$ (40)

is a standard gamma variate with parameter $\alpha=1/2$.

See also Beta Distribution, Chi-Squared Distribution

References

Beyer, W. H. CRC Standard Mathematical Tables, 28th ed. Boca Raton, FL: CRC Press, p. 534, 1987.