Random Process vs Random Variable vs Sample Space

[lahanadar]

Hi everybody,

I try to figure out connections and differences between random variables (RV), random processes (RP), and sample spaces and have confusions on some ideas you may want to help me.

All sources I searched says that RP assigns each element of a sample space to a time function. I want to give two examples:

a. Roll a six-sided die and observe the number coming out. Sample space is Ω={1,2,3,4,5,6}. Assign a time function x_i(t) from the collection {x₁(t), x₂(t), x₃(t), x₄(t), x₅(t), x₆(t)} to each ω_i∈Ω and start to roll the die to infinity. At each trial of roll, an outcome from Ω comes out, this also determines an x_i(t), and so depending on time, it generates a number. Now, is this a RP? If it is, what is the outcome of the RP?

b. My second example is on rolling a die again. This time one shot experiment is rolling a die infinitely. Eventhough sample space of one shot experiment is {1,2,3,4,5,6}, the sample space Ω subject to RP is, in fact, infinite strings where each digit of which has a number from one to six such as, (1645623...456321234...) is one possible outcome. In this case sample space Ω has infinite number of outcomes. Now I assign again each outcome to a time function x_i(t). The time function takes its domain t values such as t=1 first trial, t=2 second trial, t=i i^th trial, etc and range x_i(t) values as outcomes of the trial at time t. Assume I assigned my first trial outcome (1645623...456321234...) to x₁(t) then x₁(1)=1, x₁(2)=6, x₁(3)=4, etc. So, is this a RP? If it is, what is the outcome of this process?

Thank you for taking your time and help.

 

[Stephen Tashi]

A random process is often something that takes place in time and some books define it so it must that way. However, the most general definition of a random process is that it is "an indexed collection of random variables".

When you consider an event in reality, such as rolling dice, flipping coins etc. you shouldn't get in the habit of declaring that event is or is-not a certain type of mathematical thing. The correct view is that by defining certain aspects of the event to correspond to certain mathematical objects, you can regard the event as a certain type of mathematical thing. The same event in reality may be regarded as different mathematical things.

All sources I searched says that RP assigns each element of a sample space to a time function.

The first thing we should do is clarify if any of your sources really do say that. Can you quote what one of them actually says?

 

[lahanadar]

Sure, the book says: "A stochastic process X(t) consists of an experiment with a probability measure P[.] defined on sample space S and a function that assigns a time function x(t,s) to each outcome s in the sample space of the experiment. A stochastic process assigns a sample function to each outcome s."

When it says "A stochastic process assigns a sample function to each outcome s.", I understand that the one-shot experiment (say rolling a die) performed and an outcome came out. Since each outcome assigned to a time function by definition, an "assigned" time function also realized. Since I am doing this experiment at a time t, this time function will have a value at time t.

However from examples given in the book I understand this: A one-shot experiment (say rolling a die and assigning outcomes to random values as {even}=1 or {odd}=-1) is done repeatedly across the time which is first trial and also which creates first time function (or sequence of 1's and -1's). You complete the first experiment. Then again you perform the same one-shot experiment repeatedly across the time and generate second time function, you complete the second experiment, so on. By doing this, you have many time functions each is assigned to order of the experiment such as first experiment, second experiment, so on. At each time function, you have sequences of realizations of SAME random variable.

Therefore, I don't understand how so defined assignment business works between sample space and sample functions?
It gives these figures:

 

[Stephen Tashi]

It's interesting that when I look up stochastic process on the web, I find hits that use the same jargon "time function" as your materials. I think it is more precise to say "function of time" than a "time function". (And I prefer the definition of stochastic process that does not insist on involving time. However, let's discuss the one you are using.)

If you wanted to view throwing a single die once as a stochastic process you would need to define a process that mapped the possible outcomes to functions of time. You could define functions of time that only existed at the single point t = 0 and have the event of rolling a 3 mapped to the function X(3,t) have this function only exist at t = 0. That still leaves open the question of what the value of the function X(3,t) would be. There is no law that says X(3,0) = 3 or even that X(3,0) is a real valued function. You have a lot of freedom of choice in how you regard the throw of a single die as a stochastic process.


Suppose you want to look at repeated rolls of a single die and classify the result as even or odd. It is not correct to say that your are looking at repetitions of "the same random variable". What you mean is that your are looking at repeated samlples from independent random variables that have the same distribution. ( If they were literally "the same random variable", they would all have the same outcome. ) Let's consider the outcomes for one roll of the die to be {even, odd}. The sample space for repeated rolls of the die does not have the same outcomes. It isn't the same sample space. An outcome for repeated rolls of die is an infinite sequence. Each element of the sequence is either "even" or "odd".

In general, whenever you consider a sequence independent samples from independent, identically distributed random variables, the probability space for such sequences is not the probability space of just one of the random variables. It is a countably infinite "product" of such probability spaces.

If we look at the probability space for repeated rolls of the die as suggested above, outcomes are infinite sequences such as s = { even, even, odd, even, odd,...}. Infinite sequences have a first term , second term, etc. So if you want to define a stochastic process, one natural way would be to say that X(s,j) is the function whose value at (s,j) is the jth term of the sequence s. If we decree that the argument j represents time, then this defines X(s,t) for t = 1,2,...

I think the main source of your confusion is that your are trying to visualize the sample space involved in a stochastic process X(s,t) as something finite and easy to describe. In most practical examples, the sample space for the process is quite complicated. It's often "all possible trajectories" of some time-varying quantity. At least that makes the mapping from an outcome in the sample space to a function of time rather simple.

 

[lahanadar]

You are right, I try to visualize the sample space and stochastic process together, because books define it in that manner, however examples are not representing quite same picture. It is easy to make reasoning when you say it is indexed sequence of infinitely many random variables. However when you say to assign each outcome of sample space to a function of time, it doesn't make sense, I can't find a useful example for it. (I tried to give an example as a single roll of a die, and you positioned it to correct definition of a random process.)

From your explanations I understand,

1. The mathematical object that is assigned to a function of time is simply a trajectory (sequence), an element of "all possible trajectories". A trajectory is considered as an outcome.
2. Each point in a particular trajectory is defined by X(s,j) which is realization of a random variable at index j.

Then this example fits to our discussion: Temperature of a place is measured everyday at 10:00am for annual intervals.

a. The temperature sequence for 2011 is a function of time. All these functions could be named {..., 2010, 2011, 2012, ...}
b. Measuring temperature each day (= one-shot experiment) is a realization of a random variable W with a range of possibly all R. Domain (sample space) of this one-shot experiment for this case is again all R. (W:R→R)
c. If we fix time or index to January 1st, then this represents a random variable X(ω, Jan 1st)=X(temperature of January 1st for all years).
d. If we fix ω=2011, then this represents fixed "outcome", namely X(2011,t) which is function of time as I said at (a). Of course I am not counting 2011 as time. It identifies a particular function of time (trajectory) from others.

I hope I clarified things correctly. Thank you very much for taking your time with this. I appreciate it.

 

[Stephen Tashi]

I agree with your description of the temperature measuring process.

In applications of stochastic processes, the viewpoint that the outcomes in the sample space are trajectories is so general that almost no progress can be made studying such stochastic processes. The stochastic processes that are useful have additional properties which make them less general. The main use of the general viewpoint is to provide the background terminology needed to state the definitions of the not-so-general stochastic processes where definite results can be proven.

 

[lahanadar]

I got the idea now. In fact there is a stochastic process in a single flipping coin experiment. We flip a coin into air and wait it to rest on the floor and see what comes out, H or T. This is simply observation of an outcome from {H,T} and observation of some defined random variables {H,T}→R as well.

However, in the exactly same experiment, if we want to observe what shows up at each unit of time while the coin is traveling in the air before it rests on the floor is a stochastic process. We have to observe a sequence of combination of H's or T's each of which has an index connection to time.

One is sort of static observation and other is dynamic.

Thank you a lot for all contributions you did.