Let \(X\) be a discrete random variable with sample space \(\Omega_X\), containing only non-negative numbers. We define the PGF of \(X\), often written \(G_X\), to be equal to the power series:
\[G_X(t) = \sum_{x \in \Omega_X} \mathbb P \left({X = x}\right) t^x\]
for \(t \ge 0\). Where \(x\) and \(t\) are both zero, we take \(t^x = 1\) as is the usual convention with power series. It turns out that this function completely describes the distribution of \(X\). From this we can extract the probability that an observation of \(X\) will take a particular value, as well as the mean and variance of \(X\). In this section we will just look at how to calculate these, we'll leave PGFs of common distributions until later.
We know that: \[\sum_{x \in \Omega_X} \mathbb P \left({X = x}\right) = \mathbb P \left({X \in \Omega_X}\right) = 1\] because the value of \(X\) will always fall within \(\Omega_X\). This means that we must have \(G_X (1) = 1\). It might be the case that you're given \(G_X (t) = k f(t)\) for some normalising constant \(k\), and this property means you can substitute \(t = 1\) to quickly get \(k\) out.
With that noted, we can start deducing stuff about \(X\). By comparing the coefficient of \(t^x\) with Taylor's theorem, we have: \[\frac {{G_X}^{(x)}(0)} {x!} = \mathbb P \left({X = x}\right)\] By a similar process we can go on to compute \(\mathbb E[X]\) in terms of \(G_X (t)\) for some specific \(t\). Remember that: \[\mathbb E[X] = \sum_{x \in \Omega_X} x \mathbb P \left({X = x}\right)]\] Using a standard trick, we can differentiate \(G_X\) with respect to \(t\), [NB 1] to find: \[G'_X(t) = \sum_{x \in \Omega_X} x \mathbb P \left({X = x}\right) t^{x - 1}\] Substituting \(t = 1\) then gives us: \[G'_X(1) = \sum_{x \in \Omega_X} x \mathbb P \left({X = x}\right) = \mathbb E[X]\] Now, if we are to calculate the variance, we also need \(\mathbb E[X^2]\).
Differentiating \(G'_X\) would give: \[G''_X(t) = \sum_{x \in \Omega_X} x (x - 1) \mathbb P \left({X = x}\right) t^{x - 2}\] and so: \[G''_X(1) = \sum_{x \in \Omega_X} x (x - 1) \mathbb P \left({X = x}\right)\] Remember that: \[\mathbb E [g(X)] = \sum_{x \in \Omega_X} g(x) \mathbb P \left({X = x}\right)\] So, we can split this sum up: \[G''_X(1) = \sum_{x \in \Omega_X} x^2 \mathbb P \left({X = x}\right) - \sum_{x \in \Omega_X} x \mathbb P \left({X = x}\right) = \mathbb E[X^2] - \mathbb E[X]\] and so: \[\mathbb E[X^2] = G''_X(1) + \mathbb E[X] = G''_X(1) + G'_X(1)\] We know that: \[\operatorname{var} \left({X}\right) = \mathbb E[X^2] - (\mathbb E[X])^2 = G''_X(1) + G'_X(1) - (G'_X(1))^2\] So we have a way to find the mean and variance of a random variable, just by constructing the PGF and differentiating it. Let's move on to a common example.
S5 2005 Question 3
Let: \[G_X(t) = k[t^3 (2 + 3t) + (t + 1)^4]\]
For the second, we use: \[\mathbb E[X] = G'_X(1)\] Write the PGF as: \[G_X(t) = \frac 1 {21} [2t^3 + 3t^4 + (t+1)^4]\] Differentiating: \[G'_X(t) = \frac 1 {21} [6t^2 + 12t^3 + 4(t+1)^3]\] And substituting \(t = 1\) gives: \[\mathbb E[X] = \frac 1 {21} [6 + 12 + 4 \cdot 8] = \frac {50} {21}\]
All we need to now find the variance is to evaluate the second derivative of the pgf at \(1\): \[G''_X(t) = \frac 1 {21} [12t + 36t^2 + 12(t+1)^2]\] Substituting \(t = 1\) gives: \[G''_X(1) = \frac 1 {21} [12 + 36 + 48] = \frac {96} {21}\] And so: \[\operatorname{var}(X) = \frac {50} {21} + \frac {96} {21} - \left({\frac {96} {21}}\right)^2 = \frac {566} {441}\] There are two ways to approach the last question. First is to look at the third derivative, but in this case it's unnecessary. The more straightforward approach is to just look at the \(t^3\) term. The coefficient of the \(t^3\) term of \((1+t)^4\) is \(\binom 4 1 = \binom 4 3 = 4\) (depending on which way you expand it), so the coefficient of the \(t^3\) of the PGF is: \[\frac 1 {21} \left({2 + 4}\right) = \frac 6 {21}\] and by the definition of a pgf, we know that this is \(\mathbb P \left({X = 3}\right)\).
[NB 1: For finite \(\Omega_X\), termwise differentiation is obviously fine. For infinite \(\Omega_X\), it is not trivial, but we are allowed to differentiate convergent sums of polynomials.]
We know that: \[\sum_{x \in \Omega_X} \mathbb P \left({X = x}\right) = \mathbb P \left({X \in \Omega_X}\right) = 1\] because the value of \(X\) will always fall within \(\Omega_X\). This means that we must have \(G_X (1) = 1\). It might be the case that you're given \(G_X (t) = k f(t)\) for some normalising constant \(k\), and this property means you can substitute \(t = 1\) to quickly get \(k\) out.
With that noted, we can start deducing stuff about \(X\). By comparing the coefficient of \(t^x\) with Taylor's theorem, we have: \[\frac {{G_X}^{(x)}(0)} {x!} = \mathbb P \left({X = x}\right)\] By a similar process we can go on to compute \(\mathbb E[X]\) in terms of \(G_X (t)\) for some specific \(t\). Remember that: \[\mathbb E[X] = \sum_{x \in \Omega_X} x \mathbb P \left({X = x}\right)]\] Using a standard trick, we can differentiate \(G_X\) with respect to \(t\), [NB 1] to find: \[G'_X(t) = \sum_{x \in \Omega_X} x \mathbb P \left({X = x}\right) t^{x - 1}\] Substituting \(t = 1\) then gives us: \[G'_X(1) = \sum_{x \in \Omega_X} x \mathbb P \left({X = x}\right) = \mathbb E[X]\] Now, if we are to calculate the variance, we also need \(\mathbb E[X^2]\).
Differentiating \(G'_X\) would give: \[G''_X(t) = \sum_{x \in \Omega_X} x (x - 1) \mathbb P \left({X = x}\right) t^{x - 2}\] and so: \[G''_X(1) = \sum_{x \in \Omega_X} x (x - 1) \mathbb P \left({X = x}\right)\] Remember that: \[\mathbb E [g(X)] = \sum_{x \in \Omega_X} g(x) \mathbb P \left({X = x}\right)\] So, we can split this sum up: \[G''_X(1) = \sum_{x \in \Omega_X} x^2 \mathbb P \left({X = x}\right) - \sum_{x \in \Omega_X} x \mathbb P \left({X = x}\right) = \mathbb E[X^2] - \mathbb E[X]\] and so: \[\mathbb E[X^2] = G''_X(1) + \mathbb E[X] = G''_X(1) + G'_X(1)\] We know that: \[\operatorname{var} \left({X}\right) = \mathbb E[X^2] - (\mathbb E[X])^2 = G''_X(1) + G'_X(1) - (G'_X(1))^2\] So we have a way to find the mean and variance of a random variable, just by constructing the PGF and differentiating it. Let's move on to a common example.
S5 2005 Question 3
Let: \[G_X(t) = k[t^3 (2 + 3t) + (t + 1)^4]\]
- (a) Show that \(k = \frac 1 {21}\)
- (b) Find \(\mathbb E[X]\)
- (c) Find \(\operatorname {var} (X)\)
- (d) Find \(\mathbb P(X = 3)\)
For the second, we use: \[\mathbb E[X] = G'_X(1)\] Write the PGF as: \[G_X(t) = \frac 1 {21} [2t^3 + 3t^4 + (t+1)^4]\] Differentiating: \[G'_X(t) = \frac 1 {21} [6t^2 + 12t^3 + 4(t+1)^3]\] And substituting \(t = 1\) gives: \[\mathbb E[X] = \frac 1 {21} [6 + 12 + 4 \cdot 8] = \frac {50} {21}\]
All we need to now find the variance is to evaluate the second derivative of the pgf at \(1\): \[G''_X(t) = \frac 1 {21} [12t + 36t^2 + 12(t+1)^2]\] Substituting \(t = 1\) gives: \[G''_X(1) = \frac 1 {21} [12 + 36 + 48] = \frac {96} {21}\] And so: \[\operatorname{var}(X) = \frac {50} {21} + \frac {96} {21} - \left({\frac {96} {21}}\right)^2 = \frac {566} {441}\] There are two ways to approach the last question. First is to look at the third derivative, but in this case it's unnecessary. The more straightforward approach is to just look at the \(t^3\) term. The coefficient of the \(t^3\) term of \((1+t)^4\) is \(\binom 4 1 = \binom 4 3 = 4\) (depending on which way you expand it), so the coefficient of the \(t^3\) of the PGF is: \[\frac 1 {21} \left({2 + 4}\right) = \frac 6 {21}\] and by the definition of a pgf, we know that this is \(\mathbb P \left({X = 3}\right)\).
[NB 1: For finite \(\Omega_X\), termwise differentiation is obviously fine. For infinite \(\Omega_X\), it is not trivial, but we are allowed to differentiate convergent sums of polynomials.]
Comments
Post a Comment