Probability review

Basic probability review

  • Sample space {\Omega} can be viewed as the set of results obtained from the probabilistic experiment under consideration. e.g. If the experiment consists of flipping a coin twice, then the sample space would consist of {\{HH, HT, TT, TH \}}. Elements of {\Omega} are termed elementary events. Subsets of {\Omega} are termed events. e.g. {\{HH, HT\}} is the event that the first coin flip results in a Head.
  • Definition A {\sigma-} field {(\Omega, \mathcal{F})} consists of a sample space {\Omega} and a set {\mathcal{F}} which is a collection of subsets of {\Omega}, and satisfies the following properties:
    • {\phi \in \mathcal{F}}
    • closure under complement {\mathcal{\epsilon} \in \mathcal{F} \Rightarrow \bar{\mathcal{\epsilon}} \in \mathcal{F}}
    • closure under union { \forall i \in I, } { \mathcal{\epsilon}_i \in \mathcal{F} \Rightarrow \cup_{i \in I} \mathcal{\epsilon}_i \in \mathcal{F}}.

    Note that closure under union and closure and complement imply closure under intersection (since {A \cap B = \overline{ \bar{A} \cup \bar{B} } }). Note also that {\Omega \in \mathcal{F}}. Elements of {\mathcal{F}} can be termed as observable events, and {\mathcal{F}} can be termed as an information structure.

  • Definition Given a {\sigma-} field {(\sigma, \mathcal{F}}, a probability measure {Pr : \mathcal {F} \rightarrow \mathbb{R}^{+}} is a function which satisfies:
    • For all {\mathcal{\epsilon} \in \mathcal{F}}, {0 \leq Pr [ \mathcal{\epsilon}] \leq 1}.
    • {Pr [ \Omega] = 1}.
    • For mutually disjoint events, {\mathcal{\epsilon}_1, \mathcal{\epsilon}_2, \ldots }, {Pr [ \cup_i \mathcal{\epsilon}_i ] = \sum_{i} Pr [ \mathcal{\epsilon}_i ]}.

    Some other properties that follow from the above:

    • {Pr [ \overline{\mathop{\mathbb E}\mathcal{\epsilon}}] = 1 - Pr [ \mathcal{\epsilon}] }
    • {Pr [ A \cup B ] = Pr [ A ] + Pr [ B] - Pr [ A \cap B]}.
  • Definition A probability space {(\sigma, \mathcal{F}, Pr)} consists of a {\sigma-} field {(\sigma, \mathcal{F})} together with a probability measure {Pr} defined on it.
  • Principle of inclusion-exclusion\displaystyle  Pr [ \cup_{i=1}^{n} \mathcal{\epsilon}_i ] = \sum_i Pr [\mathcal{\epsilon}_i] - \sum_{i < j} Pr [ \mathcal{\epsilon}_i \cap \mathcal{\epsilon}_j] + \sum_{i < j < k} Pr [ \mathcal{\epsilon}_i \cap \mathcal{\epsilon}_j \cap \mathcal{\epsilon}_k] -/+ \ldots  
  • Definition Condition probability of event {A} given event {B} is defined as :\displaystyle  Pr [ A | B ] = \frac { Pr [ A \cap B ] } { Pr [B] } .

    From this, we have that : { Pr [ X \cap Y ] = Pr [ Y ] Pr [ X | Y ] = Pr [X ] Pr [Y | X ]}.

     

  • If {\mathcal{\epsilon}_1, \ldots, \mathcal{\epsilon}_k} denote a partition of the sample space {\Omega}, then, for any event {\mathcal{\epsilon}}, :\displaystyle  Pr [ \mathcal{\epsilon} ] = \sum_{i=1}^{k} Pr [\mathcal{\epsilon}_i ] Pr [ \mathcal{\epsilon} | \mathcal{\epsilon}_i] 
  • Bayes’ rule If {\mathcal{\epsilon}_1, \ldots, \mathcal{\epsilon}_k} denote a partition of the sample space {\Omega}, then, for any event {\mathcal{\epsilon}}, :\displaystyle  Pr [ \mathcal{\epsilon}_i | \mathcal{\epsilon}] = \frac { Pr [ \mathcal{\epsilon}_i] Pr [ \mathcal{\epsilon} | \mathcal{\epsilon}_i ] } { \sum_{j=1}^{k} Pr [\mathcal{\epsilon}_j] Pr [\mathcal{\epsilon} | \mathcal{\epsilon}_j]} 
  • Definition A collection of events {\{ \mathcal{\epsilon}_i | i \ in I\} } is \emph {independent} if for all subsets { S \subseteq I }, :\displaystyle  Pr [\displaystyle \cap_{i \in S} \mathcal{\epsilon}_i ] = \displaystyle \prod_{ i \in S } Pr [ \mathcal{\epsilon}_i]  
  • Equivalent defintion A collection of events {\{ \mathcal{\epsilon}_i | i \ in I\} } is \emph {independent} if for all {j \in I} and for all subsets { S \subseteq I \setminus \{j\} },\displaystyle  Pr [ \mathcal{\epsilon}_j | \displaystyle \cap_{i \in S} \mathcal{\epsilon}_i ] = Pr [ \mathcal{\epsilon}_j]
  • A collection of events {\{ \mathcal{\epsilon}_i | i \ in I\} } is \emph {k-wise independent} if every set of {k} events in the collection is independent. 2-wise independence is called pairwise independence.
  • Definition A random variable {X} is a real-valued function over the sample space {X : \Omega \rightarrow \mathbb{R}} such that for all {x \in \mathbb{R}}:\displaystyle  \{ \omega \in \Omega | X(\omega) \leq x \} \in \mathcal{F}
  • The distribution function { F : \mathbb{R} \rightarrow [0,1]} for a random variable {X} is defined as { F_X (x) = Pr [ X \leq x]}.
  • The density function {p : \mathbb{R} \rightarrow [0,1]} is defined as {p_X (x) = Pr [X = x ]}.
  • The joint distribution function and joint density function are defined likewise :\displaystyle  F_{X,Y} (x,y) = Pr [ \{X \leq x\} \cap \{ Y \leq y \} ] \displaystyle  p_{X,Y} (x,y) = Pr [ \{X =x\} \cap \{ Y = y \} ]

    .

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: