Understanding Probability Spaces: An Example Using Double Coin Tosses

[Rasalhague]

I'd like to test my understanding of some basic definitions using the example of a double coin toss. I think this would be formally modeled with the following structure:


A1. The sample space, S = {(0,0),(0,1),(1,0),(1,1)}, whose elements (s₁,s₂) are called outcomes, where s_i = 1 means heads on the ith toss, and 0 tails. (I suppose it's a configuration space, a state space whose elements are the possible final states.)

A2. The events, E. A sigma-algebra on S, containing at least all possible singletons (called elementary events).

A3. The probability, P, a probability measure on E; this P defined by P(e) = 1/4 for all singletons, e, in E.


B1. The observation space, T = {0,1,2}, whose elements, "observations", denote the possible numbers of heads achieved in two tosses.

B2. The "observed events", F. A sigma-algebra on T, containing at least all singletons.

B3. The distribution, Q, a probability measure on F.


C1. The random variable, X:S-->T such that, for all B in F, Q(B)=P(X^-1(B)).


Now, by the sigma-algebra axioms, if all singletons are in E, then so are all unions of singletons (S being countable, indeed finite), hence E is the power set of S. Since elementary events are discrete,

Q({1}) = P({(0,1)}U{(1,0)}) = P({(0,1)}) + P({(1,0)}) = 1/4 + 1/4 = 1/2

as expected. Is this how the definitions work? I'm looking to know whether I'm thinking along the right lines are if there are any flaws in my understanding of the concepts. It looks as though E and F are always power sets for a discrete probability space; is this so?

One thing that troubles me about this example is that nonzero probabilities are assigned to impossible events, i.e. events which don't correspond to one particular outcome. I hope this isn't a sign that I've got something wrong. I'm guessing it's just because the word "event" has a technical meaning here, which doesn't necessarily coincide with its everyday sense. That is, my untrained instict is to see outcome and event as synonymous, which is clearly not the case here.

 

[micromass]

Hi Rasalhague!

I'd like to test my understanding of some basic definitions using the example of a double coin toss. I think this would be formally modeled with the following structure:


A1. The sample space, S = {(0,0),(0,1),(1,0),(1,1)}, whose elements (s₁,s₂) are called outcomes, where s_i = 1 means heads on the ith toss, and 0 tails. (I suppose it's a configuration space, a state space whose elements are the possible final states.)

A2. The events, E. A sigma-algebra on S, containing at least all possible singletons (called elementary events).

A3. The probability, P, a probability measure on E; this P defined by P(e) = 1/4 for all singletons, e, in E.


B1. The observation space, T = {0,1,2}, whose elements, "observations", denote the possible numbers of heads achieved in two tosses.

B2. The "observed events", F. A sigma-algebra on T, containing at least all singletons.

B3. The distribution, Q, a probability measure on F.


C1. The random variable, X:S-->T such that, for all B in F, Q(B)=P(X^-1(B)).


Now, by the sigma-algebra axioms, if all singletons are in E, then so are all unions of singletons (S being countable, indeed finite), hence E is the power set of S. Since elementary events are discrete,

Q({1}) = P({(0,1)}U{(1,0)}) = P({(0,1)}) + P({(1,0)}) = 1/4 + 1/4 = 1/2

as expected. Is this how the definitions work?

Indeed, it appears you've understood things quite well! Denote that you've only given P and Q on the singletons, it would be a good exercise to give P and Q of some other events...

I'm looking to know whether I'm thinking along the right lines are if there are any flaws in my understanding of the concepts. It looks as though E and F are always power sets for a discrete probability space; is this so?

Yes, when working with discrete spaces, you usually put the power set as sigma-algebra. In particular, if you want the singletons to be in the sigma-algebra, then you will always end up with the power set!
When working with continuous spaces, the power set will be too big and you will have to use something called the Borel sigma algebra.

One thing that troubles me about this example is that nonzero probabilities are assigned to impossible events, i.e. events which don't correspond to one particular outcome. I hope this isn't a sign that I've got something wrong.

Can you give an example of this? This shouldn't happen in this case, and I do fear you misunderstand something then...

[/QUOTE]
I'm guessing it's just because the word "event" has a technical meaning here, which doesn't necessarily coincide with its everyday sense. That is, my untrained instict is to see outcome and event as synonymous, which is clearly not the case here.[/QUOTE]

Well, outcome and event are certainly not synonymous. Maybe the choice for the word "event" is not a good one, but there's no use in changing it. An outcome is what you expect it to be, while an event is a set of outcomes. For example {(1,0),(0,1)} is an event and it denotes that we either have (1,0) or (0,1) as outcome.
Maybe your flaw is that you interpret this as "both (1,0) and (0,1) happen", which is not the way you should do it. You should interpret it as "we have (1,0) or (0,1)", or perhaps as: "we have exactly one head and one tail".

 

[Rasalhague]

Hi Rasalhague!

Indeed, it appears you've understood things quite well! Denote that you've only given P and Q on the singletons, it would be a good exercise to give P and Q of some other events...

Hi micromass! And thanks for your reassuring reply! I'm fairly confident about how to get the other values of P and Q.

For example, Q({0,1}) =Q(F) - Q({0,1}^C) = Q(F) - Q({2}) = 1 - Q({2}) = 1 - P({(1,1)}) = 1 - 1/4 = 3/4, where superscript C denotes complementation. This because, if B is in F, then Q(F) = 1, one of the properties of a probability measure, and Q(B) + Q(B^C) = Q(B U B^C), according to the additive property of a probability measure.

I'm not sure what other example to pick, that would show anything new. Here's an example of a probability value "associated with" the sample space but not the observation space: P({(0,1),(1,1)}) = P({(0,1) U (1,1)}) = P({(0,1)}) + P({(1,1)}) = 1/4 + 1/4 = 1/2, just as in the example from my first post.

Yes, when working with discrete spaces, you usually put the power set as sigma-algebra. In particular, if you want the singletons to be in the sigma-algebra, then you will always end up with the power set!
When working with continuous spaces, the power set will be too big and you will have to use something called the Borel sigma algebra.

Great, it's all starting to make sense at last! Do you know any example of an application where another sigma-algebra is used instead of the power set? Is the name "discrete sigma-algebra" given to the power set? I'm thinking of the analogy with discrete topology.

Can you give an example of this? This shouldn't happen in this case, and I do fear you misunderstand something then...

Well, outcome and event are certainly not synonymous. Maybe the choice for the word "event" is not a good one, but there's no use in changing it. An outcome is what you expect it to be, while an event is a set of outcomes. For example {(1,0),(0,1)} is an event and it denotes that we either have (1,0) or (0,1) as outcome.
Maybe your flaw is that you interpret this as "both (1,0) and (0,1) happen", which is not the way you should do it. You should interpret it as "we have (1,0) or (0,1)", or perhaps as: "we have exactly one head and one tail".

Yes, that's exactly the flaw; although I was really just checking that I shouldn't be interpreting it in that tempting way. By "impossible events", I meant non-elementary events. But I see now, from what you write, that this isn't a problem.

 

[micromass]

Great, it's all starting to make sense at last! Do you know any example of an application where another sigma-algebra is used instead of the power set?

I can, but I fear this will just confuse you. Just wait a little while and you'll see these things popping up soon. When working with uncountable sets, we often do not use the power set, but the Borel sigma-algebra. You'll see this in due time!

Is the name "discrete sigma-algebra" given to the power set? I'm thinking of the analogy with discrete topology.

Hmmm, I like the name discrete sigma-algebra. Sadly, it is not standard terminology, in that, I've never seen it being used before...

 

[Rasalhague]

Hmmm, I like the name discrete sigma-algebra. Sadly, it is not standard terminology, in that, I've never seen it being used before...

Google finds 28 examples. One gives 2^V x 2^V as the discrete sigma-algebra on V x V. The rest seem to take it for granted the reader will know what is meant.

 

[Rasalhague]

I can, but I fear this will just confuse you. Just wait a little while and you'll see these things popping up soon. When working with uncountable sets, we often do not use the power set, but the Borel sigma-algebra. You'll see this in due time!

Okay, fair enough! Thanks for all your help and encouragement.

 

[bpet]

... Do you know any example of an application where another sigma-algebra is used instead of the power set? Is the name "discrete sigma-algebra" given to the power set? I'm thinking of the analogy with discrete topology...

An artificial example would be {{},{1,2},{3,4},{1,2,3,4}} where no singletons are measurable but these don't tend to occur in practise because usually the fundamental events are modeled as singletons.

Also your "observation" T is more commonly known as a random variable. A function S defined on the above example by S(w)=w is not a random variable, because e.g. the event {S<=1} is not measurable.

 

[Rasalhague]

Also your "observation" T is more commonly known as a random variable.

T, in my posts, denotes the observation space. I informally called its elements "observations", but I don't know if that's a standard term. It seemed a natural name, but obviously I'd better not do this if people treat "observation" as a synonym for "observation space". Is that the case?

Is your point that it's more common to give the name random variable to a letter denoting possible values of my function X, rather than to the function itself. I've been provisionally thinking of the notation P(X<5) as an example of the mathematical tradition of informally conflating a function with its value, as when people write y = y(x), and say "y is a function of x".

 

[Rasalhague]

An artificial example would be {{},{1,2},{3,4},{1,2,3,4}} where no singletons are measurable but these don't tend to occur in practise because usually the fundamental events are modeled as singletons.

Also your "observation" T is more commonly known as a random variable. A function S defined on the above example by S(w)=w is not a random variable, because e.g. the event {S<=1} is not measurable.

Isn't the identity function always measurable, providing the same sigma algebra is used? What do you make of the following argument? The pre-image of every measurable set is itself. So if a set is measurable, its pre-image under the identity function will also be measurable. In your example, the set {n in {1,2,3,4} | n <=1} = {1} is not an element of the sigma algebra you proposed, so the fact that its pre-image under the identity function is not an element of this sigma algebra is not a proof that the identity function is not measurable.