Difference between revisions of "Expectation of the geometric distribution"

Revision as of 23:27, 30 November 2017 (view source) Alec (Talk | contribs) (Saving work)		Latest revision as of 02:55, 16 January 2018 (view source) Alec (Talk | contribs) (Finishing proof, there's case extension to be done as there's no reason it can't cover p in [0,1) for case 2, rather than just (0,1) as it does now)
Line 1:		Line 1:
	{{Stub page|grade=A|msg=Finish in morning}}		{{Stub page|grade=A|msg=Finish in morning}}
−	{{ProbMacros}}	+	{{ProbMacros}}{{M|\newcommand{\d}[0]{\mathrm{d} } }}
	__TOC__		__TOC__
	==Statement==		==Statement==
Line 33:		Line 33:
	We now consider the {{M|S'_n}} terms:		We now consider the {{M|S'_n}} terms:
	* {{MM|S'_n\eq pS_n\eq p\left(\sum^n_{k\eq 1}kq^{k-1}\right)}} - {{0^0}} comes up here		* {{MM|S'_n\eq pS_n\eq p\left(\sum^n_{k\eq 1}kq^{k-1}\right)}} - {{0^0}} comes up here
−	===Case 2: {{M|p\in (0,1)\subseteq\mathbb{R} }}===	+	===Case 2: {{M|p\in (0,1)\subseteq\mathbb{R} }} - {{XXX|EXTENSION}}===
−	Here we use:	+	: {{XXX|This case can be extended to {{M|p\in (0,1]}} and should be}} as there's no reason we can't cope with the {{M|q\eq 0}} case - {{XXX|SORT THIS OUT}}
−	* {{MM|\frac{\mathrm{d} }{\mathrm{d}q}\Big[q^k\Big]\Bigg\vert_q\eq kq^{k-1} }} and then the {{MM|\sum^n_{k\eq 1}q^k}} is a [[geometric series]] - starting at {{M|q}} though not {{M|1}}	+	'''[[Lemmas|Lemmas used]]:'''
		+	# {{MM|\frac{\mathrm{d} }{\mathrm{d}q}\Big[q^k\Big]\Bigg\vert_q\eq kq^{k-1} }} which is the first result covered in [[differentiation]]<ref group="Note">I'd really like to link to something here so {{XXX|Link to the actual result!}}</ref>
		+	# {{MM|\sum^n_{k\eq 1}r^{k-1}\eq \frac{1-r^n}{1-r} }} (from the result on the ''[[geometric series]]'' page), and,
		+	#* Note that {{MM|\sum_{k\eq 1}^nr^k\eq r\sum^n_{k\eq 1}r^{k-1} }} so we will really use:
		+	#** {{MM|\sum^n_{k\eq 1}r^k\eq r\frac{1-r^n}{1-r} }}
		+
		+
		+	'''Proof:'''
		+	* Let {{M|p\in}}[[open interval|{{M|(0,1)}}]]{{M|\subseteq}}[[Reals|{{M|\mathbb{R} }}]] be given, and let {{M|X\sim}}[[Geometric distribution|{{M|\text{Geo} }}]]{{M|(p)}} so {{M|\P{X\eq k}:\eq (1-p)^{k-1}p}} for {{M|k\in\mathbb{N}_{\ge 1} }}, now:
		+	** {{MM|\E{X}:\eq\sum^\infty_{k\eq 1}k\cdot\P{X\eq k}\eq \sum^\infty_{k\eq 1}k(1-p)^{k-1}p\eq p\sum^\infty_{k\eq 1}kq^{k-1} }} where we have substituted {{M|q^{k-1} }} for {{M|(1-p)^{k-1} }} at the end there.
		+	*** We use the first lemma described above to observe that {{MM|kq^{k-1}\eq\frac{\d }{\d q}\Big[q^k\Big]\Big\vert_q}}, thus:
		+	**** {{MM|\E{X}\eq p\sum^\infty_{k\eq 1}\frac{\d}{\d q}\Big[q^k\Big]\Big\vert_q}}
		+	****: {{MM|\eq p\cdot\left(\frac{\d}{\d q}\left[\sum^\infty_{k\eq 1}q^k\middle]\right\vert_q\right)}}<ref group="Note" name="CalculusSum"/>
		+	** We now work on the expression: {{MM|\sum^\infty_{k\eq 1}q^k}}, taking it as {{MM|\lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}q^k\right) }} and operate on the {{MM|\sum^n_{k\eq 1}q^k}} first
		+	*** By the ''second lemma'' above:
		+	**** {{MM|\sum^n_{k\eq 1}q^k\eq q\frac{1-q^n}{1-q} }}
		+	****: {{MM|\eq \frac{q}{1-q}\cdot(1-q^n) }}
		+	*** Now we consider {{MM|\lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}q^k\right) }},
		+	**** {{MM|\lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}q^k\right) \eq \lim_{n\rightarrow\infty}\left(\frac{q}{1-q}\cdot(1-q^n) \right) }}
		+	****: {{MM|\eq\frac{q}{1-q}\cdot\lim_{n\rightarrow\infty}\Big((1-q^n)\Big) }}
		+	**** Let us operate on the {{MM|\lim_{n\rightarrow\infty}\Big((1-q^n)\Big) }} now
		+	***** We have three cases, {{M|q\eq 0}}, {{M|q\in(0,1)}} and {{M|q\eq 1}} - {{Notice|But as we are explicitly in the {{M|q\in(0,1)}} case we don't need to consider them really, we do so for demonstration purposes only}}
		+	****: All of these are applications of [[limit of integer powers of a real value]]
		+	****:# {{M|q\eq 0}} then obviously {{M|0^n}} for {{M|n\in\mathbb{N}_{\ge 1} }} is always {{M|0}} (with [[0^0|{{M|0^0}}]] "disputed" but not relevant here) so
		+	****:#* {{M|\lim_{n\rightarrow\infty}(1-q^n)\eq 1-0\eq 1}}
		+	****:# {{M|q\in(0,1)}} then {{M|q^n}} gets smaller as {{M|n}} increases so {{M|q^n\rightarrow 0}} so {{M|1-q^n\rightarrow 1}}, thus
		+	****:#* {{M|\lim_{n\rightarrow\infty}(1-q^n)\eq 1-0\eq 1}} also
		+	****:# {{M|q\eq 1}} then {{M|q^n\eq 1}} always so
		+	****:#* {{M|\lim_{n\rightarrow\infty}(1-q^n)\eq 1-1\eq 0}}
		+	***** So we see that {{M|q\in [0,1) }}<ref group="Note">I'm extending the range slightly but as {{M|(0,1)\subseteq [0,1)}} we're fine to do so</ref> means that {{M|\lim_{n\rightarrow\infty}(1-q^n)\eq 1}}
		+	**** Substituting our findings we see for the relevant range of this case that:
		+	***** {{MM|\frac{q}{1-q}\cdot\lim_{n\rightarrow\infty}\Big((1-q^n)\Big) \eq \frac{q}{1-q} }}
		+	**** Thus:
		+	***** {{MM|\sum^\infty_{k\eq 1}q^k:\eq \lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}q^k\right) \eq \frac{q}{1-q} }}
		+	** We combine this into our expression for {{M|\E{X} }}:
		+	*** {{MM|\E{X}\eq p\cdot\left(\frac{\d}{\d q}\left[\sum^\infty_{k\eq 1}q^k\middle]\right\vert_q\right)}}
		+	***: {{MM|\eq p\cdot\frac{\d}{\d q}\left[\frac{q}{1-q}\middle]\right\vert_q}}
		+	*** We now operate on {{M|\frac{\d}{\d q}\big[q\cdot (1-q)^{-1}\big]\big\vert_q }} and - as writing it this way implies - will use the [[product rule]]:
		+	**** {{MM|\frac{\d}{\d q}\Big[q\cdot (1-q)^{-1}\Big]\Big\vert_q \eq q\frac{\d}{\d q}\left[(1-q)^{-1}\middle]\right\vert_q+\frac{1}{1-q}\frac{\d}{\d q}\left[q\right]\Big\vert_q}}
		+	****: {{MM|\eq\frac{-q}{(1-q)^2}\cdot\frac{\d}{\d q}\Big[(1-q)\Big]\Big\vert_q +\frac{1}{1-q} }} - notice the [[chain rule|chain ruling]] being applied here
		+	****: {{MM|\eq\frac{-q}{(1-q)^2}(-1) +\frac{1}{1-q} }}
		+	****: {{MM|\eq \frac{1}{1-q}\left(1+\frac{q}{1-q}\right) }}
		+	****: {{MM|\eq \frac{1}{1-q}\left(\frac{1-q}{1-q}+\frac{q}{1-q}\right) }}
		+	****: {{MM|\eq \frac{1}{1-q}\left(\frac{1}{1-q}\right) }}
		+	****: {{MM|\eq\frac{1}{(1-q)^2} }} or {{M|\eq (1-q)^{-2} }}
		+	**** Finally: {{MM|\frac{\d}{\d q}\big[q\cdot (1-q)^{-1}\big]\big\vert_q \eq \frac{1}{(1-q)^2} }}
		+	***** So now we have: {{MM|\E{X} \eq p\cdot\frac{\d}{\d q}\left[\frac{q}{1-q}\middle]\right\vert_q\eq p\frac{1}{(1-q)^2} }}
		+	*** Lastly we operate on {{MM|\E{X}\eq p\frac{1}{(1-q)^2} }}
		+	**** Recall that {{M|q:\eq 1-p}} so {{M|1-q\eq 1-(1-p)\eq 1-1+p\eq p}} - we have {{M|1-q\eq p}} now, substitute this in and we see:
		+	***** {{MM|\E{X}\eq p\frac{1}{p^2} }}
		+	*****: {{MM|\eq \frac{1}{p} }}
		+	** So we have {{MM|\E{X}\eq \frac{1}{p} }} given our value of {{M|p}}
		+	* Since our choice of {{M|p\in(0,1)}} was arbitrary we have shown:
		+	** {{MM|\forall p\in(0,1)\left[\E{\text{Geo}(p)}\eq\frac{1}{p}\right] }} - as required
	==Notes==		==Notes==
−	<references group="Note"/>	+	<references group="Note">
−	{{Theorem Of|Probability|Elementary Probability|Statistics}}[[Category:Variance Calculations]]	+	<ref group="Note" name="CalculusSum">Remember {{MM|\sum^\infty_{k\eq 1}a_k}} is just short hand for {{MM|\lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}a_k\right)}} - see [[limits]] and [[limit of a series]] - and remember that {{MM|\frac{\d}{\d x}\Big[f(x)\Big]\Big\vert_x+\frac{\d}{\d x}\Big[g(x)\Big]\Big\vert_x\eq\frac{\d}{\d x}\Big[f(x)+g(x)\Big]\Big\vert_x}} - as per [[linearity of the derivative]].
		+	* {{XXX|More links?}}</ref>
		+	</references>
		+	{{Theorem Of|Probability|Elementary Probability|Statistics}}[[Category:Expectation Calculations]]

---

Latest revision as of 02:55, 16 January 2018

$\newcommand{\E}[1]{ {\mathbb{E}{\left[{#1}\right]} } }$$\newcommand{\Mdm}[1]{\text{Mdm}{\left({#1}\right) } }$$\newcommand{\Var}[1]{\text{Var}{\left({#1}\right) } }$$\newcommand{\ncr}[2]{ \vphantom{C}^{#1}\!C_{#2} }$$\newcommand{\d}[0]{\mathrm{d} }$

Statement

Let $X\sim$$\text{Geo}$$(p)$ where $p$ is the probability of any trial being a success, and each trial is i.i.d as $X_i\sim$$\text{Borv}$$(p)$, from this we have:

• For $k\in\mathbb{N}_{\ge 1}$ that $\P{X\eq k}\eq p(1-p)^{k-1}$

We now define $q:\eq 1-p$ as this will simplify calculations further on, meaning that now:

• For $k\in\mathbb{N}_{\ge 1}$ that $\P{X\eq k}\eq pq^{k-1}$

Expectation

• The expectation of $X$ is:
  • $$\sum^\infty_{k\eq 1}k\P{X\eq k}$$ which of course is actually a limit of a series, $$\lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}k\P{X\eq k}\right)$$

We claim that that $\E{X}\eq\frac{1}{p}$ for $p\in$$(0,1]$$\subseteq\mathbb{R}$ and undefined for $p\eq 0$

To do so we will consider the 3 cases, $p\eq 0$, $p\in (0,1)\subseteq\mathbb{R}$ and $p\eq 1$ separately and in reverse of this order.

See also

• Variance of the geometric distribution
• Mdm of the geometric distribution

Proof

We introduce the following for short.

1. $$S'_n:\eq\sum^n_{k\eq 1}kpq^{k-1}$$ - this forms the sequence used in the limit - which is a series.
   • Thus $$\E{X}\eq\lim_{n\rightarrow\infty}\Big(S'_n\Big)$$
2. $$S_n:\eq\sum^n_{k\eq 1}kq^{k-1}$$
   • This comes from the sequence inside the limit, $$\sum^n_{k\eq 1}k\P{X\eq k}\eq\sum^n_{k\eq 1}kpq^{k-1}\eq p\sum^n_{k\eq 1} kq^{k-1} \eq pS_n$$ , so:
     • $$\E{X}\eq\lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}k\P{X\eq k}\right)\eq\lim_{n\rightarrow\infty}\Big(pS_n\Big)$$

Notice that $S'_n\eq pS_n$ - introduced purely to save typing.

Case 1: $p\eq 1$

Notice that in this case, $q\eq 1-p\eq 0$.

We now consider the $S'_n$ terms:

• $$S'_n\eq pS_n\eq p\left(\sum^n_{k\eq 1}kq^{k-1}\right)$$ - $0^0$ comes up here

Case 2: $p\in (0,1)\subseteq\mathbb{R}$ -

TODO: EXTENSION

TODO: This case can be extended to $p\in (0,1]$ and should be

as there's no reason we can't cope with the $q\eq 0$ case -

TODO: SORT THIS OUT

Lemmas used:

1. $$\frac{\mathrm{d} }{\mathrm{d}q}\Big[q^k\Big]\Bigg\vert_q\eq kq^{k-1}$$ which is the first result covered in differentiation^{[Note 1]}
2. $$\sum^n_{k\eq 1}r^{k-1}\eq \frac{1-r^n}{1-r}$$ (from the result on the geometric series page), and,
   • Note that $$\sum_{k\eq 1}^nr^k\eq r\sum^n_{k\eq 1}r^{k-1}$$ so we will really use:
     • $$\sum^n_{k\eq 1}r^k\eq r\frac{1-r^n}{1-r}$$

Proof:

• Let $p\in$$(0,1)$$\subseteq$$\mathbb{R}$ be given, and let $X\sim$$\text{Geo}$$(p)$ so $\P{X\eq k}:\eq (1-p)^{k-1}p$ for $k\in\mathbb{N}_{\ge 1}$, now:
  • $$\E{X}:\eq\sum^\infty_{k\eq 1}k\cdot\P{X\eq k}\eq \sum^\infty_{k\eq 1}k(1-p)^{k-1}p\eq p\sum^\infty_{k\eq 1}kq^{k-1}$$ where we have substituted $q^{k-1}$ for $(1-p)^{k-1}$ at the end there.
    • We use the first lemma described above to observe that $$kq^{k-1}\eq\frac{\d }{\d q}\Big[q^k\Big]\Big\vert_q$$ , thus:
      • $$\E{X}\eq p\sum^\infty_{k\eq 1}\frac{\d}{\d q}\Big[q^k\Big]\Big\vert_q$$

        $$\eq p\cdot\left(\frac{\d}{\d q}\left[\sum^\infty_{k\eq 1}q^k\middle]\right\vert_q\right)$$ ^{[Note 2]}

  • We now work on the expression: $$\sum^\infty_{k\eq 1}q^k$$ , taking it as $$\lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}q^k\right)$$ and operate on the $$\sum^n_{k\eq 1}q^k$$ first
    • By the second lemma above:
      • $$\sum^n_{k\eq 1}q^k\eq q\frac{1-q^n}{1-q}$$

        $$\eq \frac{q}{1-q}\cdot(1-q^n)$$

    • Now we consider $$\lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}q^k\right)$$ ,
      • $$\lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}q^k\right) \eq \lim_{n\rightarrow\infty}\left(\frac{q}{1-q}\cdot(1-q^n) \right)$$

        $$\eq\frac{q}{1-q}\cdot\lim_{n\rightarrow\infty}\Big((1-q^n)\Big)$$

      • Let us operate on the $$\lim_{n\rightarrow\infty}\Big((1-q^n)\Big)$$ now
        • We have three cases, $q\eq 0$, $q\in(0,1)$ and $q\eq 1$ - But as we are explicitly in the $q\in(0,1)$ case we don't need to consider them really, we do so for demonstration purposes only

        All of these are applications of limit of integer powers of a real value

        1. $q\eq 0$ then obviously $0^n$ for $n\in\mathbb{N}_{\ge 1}$ is always $0$ (with $0^0$ "disputed" but not relevant here) so
           • $\lim_{n\rightarrow\infty}(1-q^n)\eq 1-0\eq 1$
        2. $q\in(0,1)$ then $q^n$ gets smaller as $n$ increases so $q^n\rightarrow 0$ so $1-q^n\rightarrow 1$, thus
           • $\lim_{n\rightarrow\infty}(1-q^n)\eq 1-0\eq 1$ also
        3. $q\eq 1$ then $q^n\eq 1$ always so
           • $\lim_{n\rightarrow\infty}(1-q^n)\eq 1-1\eq 0$

        • So we see that $q\in [0,1)$^{[Note 3]} means that $\lim_{n\rightarrow\infty}(1-q^n)\eq 1$
      • Substituting our findings we see for the relevant range of this case that:
        • $$\frac{q}{1-q}\cdot\lim_{n\rightarrow\infty}\Big((1-q^n)\Big) \eq \frac{q}{1-q}$$
      • Thus:
        • $$\sum^\infty_{k\eq 1}q^k:\eq \lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}q^k\right) \eq \frac{q}{1-q}$$
  • We combine this into our expression for $\E{X}$:
    • $$\E{X}\eq p\cdot\left(\frac{\d}{\d q}\left[\sum^\infty_{k\eq 1}q^k\middle]\right\vert_q\right)$$

      $$\eq p\cdot\frac{\d}{\d q}\left[\frac{q}{1-q}\middle]\right\vert_q$$

    • We now operate on $\frac{\d}{\d q}\big[q\cdot (1-q)^{-1}\big]\big\vert_q$ and - as writing it this way implies - will use the product rule:
      • $$\frac{\d}{\d q}\Big[q\cdot (1-q)^{-1}\Big]\Big\vert_q \eq q\frac{\d}{\d q}\left[(1-q)^{-1}\middle]\right\vert_q+\frac{1}{1-q}\frac{\d}{\d q}\left[q\right]\Big\vert_q$$

        $$\eq\frac{-q}{(1-q)^2}\cdot\frac{\d}{\d q}\Big[(1-q)\Big]\Big\vert_q +\frac{1}{1-q}$$ - notice the chain ruling being applied here

        $$\eq\frac{-q}{(1-q)^2}(-1) +\frac{1}{1-q}$$

        $$\eq \frac{1}{1-q}\left(1+\frac{q}{1-q}\right)$$

        $$\eq \frac{1}{1-q}\left(\frac{1-q}{1-q}+\frac{q}{1-q}\right)$$

        $$\eq \frac{1}{1-q}\left(\frac{1}{1-q}\right)$$

        $$\eq\frac{1}{(1-q)^2}$$ or $\eq (1-q)^{-2}$

      • Finally: $$\frac{\d}{\d q}\big[q\cdot (1-q)^{-1}\big]\big\vert_q \eq \frac{1}{(1-q)^2}$$
        • So now we have: $$\E{X} \eq p\cdot\frac{\d}{\d q}\left[\frac{q}{1-q}\middle]\right\vert_q\eq p\frac{1}{(1-q)^2}$$
    • Lastly we operate on $$\E{X}\eq p\frac{1}{(1-q)^2}$$
      • Recall that $q:\eq 1-p$ so $1-q\eq 1-(1-p)\eq 1-1+p\eq p$ - we have $1-q\eq p$ now, substitute this in and we see:
        • $$\E{X}\eq p\frac{1}{p^2}$$

          $$\eq \frac{1}{p}$$

  • So we have $$\E{X}\eq \frac{1}{p}$$ given our value of $p$
• Since our choice of $p\in(0,1)$ was arbitrary we have shown:
  • $$\forall p\in(0,1)\left[\E{\text{Geo}(p)}\eq\frac{1}{p}\right]$$ - as required

Notes

1. Jump up ↑ I'd really like to link to something here so
   TODO: Link to the actual result!
2. Jump up ↑ Remember $$\sum^\infty_{k\eq 1}a_k$$ is just short hand for $$\lim_{n\rightarrow\infty}\left(\sum^n_{k\eq 1}a_k\right)$$ - see limits and limit of a series - and remember that $$\frac{\d}{\d x}\Big[f(x)\Big]\Big\vert_x+\frac{\d}{\d x}\Big[g(x)\Big]\Big\vert_x\eq\frac{\d}{\d x}\Big[f(x)+g(x)\Big]\Big\vert_x$$ - as per linearity of the derivative.
   • TODO: More links?
3. Jump up ↑ I'm extending the range slightly but as $(0,1)\subseteq [0,1)$ we're fine to do so