Retrodictive Inferences in Quantum Mechanics

[Morbert]

TL;DR Summary

While time asymmetry of a measurement process restricts retrodiction (the inference of previous measurement outcomes, counterfactual or otherwise, based on current measurement outcomes), it does not rule them out. It only rules out retrodiction asymptotically into the past, and we can readily construct a time-symmetric theory of measurement and identify where and when retrodictive inferences are valid. The meaning of these inferences are interpretation-dependent.

Take a simple case: A system is prepared in state ##\rho_i## at time ##t_0##, and a projective measurement is performed at time ##t_2## with an outcome ##b##. We can retrodict a projective measurement outcome ##a## at time ##t_1## where ##t_0<t_1<t_2##$$p(a|b) = \frac{\mathrm{Tr}\left[\Pi_b(t_2)\Pi_a(t_1)\rho_i\Pi_a(t_1)\right]}{\sum_{a'}\mathrm{Tr}\left[\Pi_b(t_2)\Pi_{a'}(t_1)\rho_i\Pi_{a'}(t_1)\right]}$$More generally, we can retrodict a sequence of measurement outcomes ##\alpha = i,j,\dots m## $$p(\alpha|b) = \frac{\mathrm{Tr}\left[\Pi_b(t_2)C^\dagger_\alpha\rho_iC_\alpha\right]}{\sum_{\alpha'}\mathrm{Tr}\left[\Pi_b(t_2)C^\dagger_{\alpha'}\rho_iC_{\alpha'}\right]}$$where ##C_\alpha = \Pi_i\Pi_j\dots\Pi_m##. It can be helpful to interpret the ##b## measurement as part of the preparation process: a post-selection of a final characteristic, such that e.g. the ensemble of experimental runs to be interrogated is characterised by ##\rho_i,\rho_f## and $$p(\alpha;\rho_i,\rho_f) = \mathrm{Tr}\left[\rho_fC^\dagger_\alpha\rho_iC_\alpha\right]$$This expression is time-symmetric. We can reverse the sequence of observations and preparation procedures and the expression won't change.

These computations sometimes let us seemingly violate the Heisenberg uncertainty principle. Take for example the case where a particle is prepared by preselecting for spin-x = ##\uparrow## at ##t_0## and post-selecting for spin-z = ##\uparrow## at ##t_2##. We can see that a measurement for either spin-x or spin-z at ##t_1## will have yielded the outcome ##\uparrow## with probability $$p(x_\uparrow;\rho_i,\rho_f) = p(z_\uparrow;\rho_i,\rho_f) = 1$$ Certainty about spin-x and spin-z at the same time?? These probabilities are reliable, because we aren't explicitly constructing some outcome ##c## where ##\Pi_c = \Pi_{x_\uparrow}(t_1)\Pi_{z_\uparrow}(t_1)## which has no Hilbert subspace. The complementary measurements can never be performed on the same system.

The meaning of these retrodictions are interpretation-dependent. Ensemble interpretations are the most straightforward. If you present me with the ensemble in the last paragraph, I can know that for each member of the ensemble, if you have measured spin-x at ##t_1## you will have observed ##\uparrow## and if you have measured spin-z you will have observed ##\uparrow##.

 

[PeterDonis]

We can see that a measurement for either spin-x or spin-z at ##t_1## will have yielded the outcome ##\uparrow## with probability $$p(x_\uparrow;\rho_i,\rho_f) = p(z_\uparrow;\rho_i,\rho_f) = 1$$

While this is true, we have to be very careful in drawing inferences from it. For example, we cannot draw this inference:

Certainty about spin-x and spin-z at the same time??

Why not? Because it is impossible to measure both spin-x and spin-z at time ##t_1##. You can only measure one or the other. And whichever one you actually measure, you will not be certain of the other at time ##t_1##. If you measure spin-x up at time ##t_1##, then spin-z is uncertain at time ##t_1##; and if you measure spin-z up at time ##t_1##, then spin-x is uncertain at time ##t_1##. And if you don't measure either one at time ##t_1##, then your retrodiction is irrelevant because you can't conclude anything from a retrodiction about a measurement that was never actually made.

 

[CelHolo]

This is ultimately related I think to the ambiguity of defining general conditional Probabilities in quantum theory. If we make ##n## measurements with the same set of possible outcomes ##j = 1 \ldots m## and denote the ##i##-th experiment having outcome ##j## by ##E^{j_{i}}_{i}## then classical probability tells us:
##P(E^{j_{i}}_{i}|E_{1}^{j_{1}},\ldots , E_{i-1}^{j_{i-1}},E_{i+1}^{j_{i+1}},\ldots,E_{n}^{j_{n}}) = \frac{P(E_{1}^{j_{1}}\ldots E^{j_{i}}_{i},\ldots,E_{n}^{j_{n}})}{P(E_{1}^{j_{1}}\ldots E^{j_{i}}_{i},\ldots,E_{n}^{j_{n}}) + P(E_{1}^{j_{1}}\ldots \lnot E^{j_{i}}_{i},\ldots,E_{n}^{j_{n}})}##
where ##\lnot E^{j_{i}}_{i}## is the event that the ##i##-th experiment results in any outcome other than outcome ##j##.
This is essentially the probability that experiment ##i## had some outcome ##j## given that we know all the outcomes for the other experiments, so it's a form of retrodiction.

In classical probability theory ##E^{j_{i}}_{i}## is just some subset of the probability space ##\Omega## and thus ##\lnot E^{j_{i}}_{i}## is just defined as its complement.

In quantum probability however ##E^{j_{i}}_{i}## is represented by some projector or povm, but there is no unique definition for ##\lnot E^{j_{i}}_{i}##. Taking a one-dimensional projector on a ##d##-dimensional Hilbert space for simplicity, ##\lnot E^{j_{i}}_{i}## could any collection of ##d-1## one dimensional projectors onto orthogonal lines. These different ways of representing ##\lnot E^{j_{i}}_{i}## do not give the same probability in the denominator above.

To define the conditional probability for a retrodiction you have to specify what set of POVMs at time ##i## that ##E^{j_{i}}_{i}## was drawn from, so that you can select a specific representation for ##\lnot E^{j_{i}}_{i}##. In other words to retrodict you have to know what measurement was performed. This "block" in defining a conditional is equivalent to the Kochen-Specker theorem for anybody interested.

For retrodictions of macroscopic collective coordinates based on observations of other such collective coordinates one can show that the different formulations of ##\lnot E^{j_{i}}_{i}## give the same probability (up to error terms of ##\mathcal{O}(10^{-10^{100}})## or smaller). So thankfully to talk about the chance a Brachiosaurus died in a certain way from looking at its fossils we don't need to consider what possible measurement was made on the Brachiosaurus back in the Jurassic! However for atomic scale processes we unfortunately cannot retrodict without considering the method of data acquisition.

 

[Morbert]

we have to be very careful in drawing inferences from it [...] it is impossible to measure both spin-x and spin-z at time ##t_1##. You can only measure one or the other. And whichever one you actually measure, you will not be certain of the other at time ##t_1##

This is what I tried to get across here

These probabilities are reliable, because we aren't explicitly constructing some outcome ##c## where ##\Pi_c = \Pi_{x_\uparrow}(t_1)\Pi_{z_\uparrow}(t_1)## which has no Hilbert subspace. The complementary measurements can never be performed on the same system.

alternatively, we say the probabilities ##p(x_\uparrow), p(z_\uparrow)## reference complementary sample spaces.

But we can still make inferences not permitted by more typical prediction scenarios. E.g. Say Vaidman, up to his old tricks, attaches a bomb to each member of an ensemble ##\rho_i## that performs a POVM ##\{E_\uparrow, E_\downarrow\}## at ##t_1## where ##E_{\uparrow(\downarrow)} = \frac{1}{2}\left(\Pi_{x_{\uparrow(\downarrow)}} + \Pi_{z_{\uparrow(\downarrow)}}\right)##. I.e. the Bomb performs a spin measurement on an undetermined axis x or z. If it registers the outcome ##\downarrow##, it explodes. I can be certain that no bombs connected to any member of the ensemble characterised by the pre- and post-selection ##\rho_i,\rho_f## exploded.

tl;dr I agree that we have to be careful in our inferences, but we also should be careful not to suffer from an abundance of caution.

 

[PeterDonis]

But we can still make inferences not permitted by more typical prediction scenarios. E.g. Say Vaidman, up to his old tricks, attaches a bomb to each member of an ensemble ##\rho_i## that performs a POVM ##\{E_\uparrow, E_\downarrow\}## at ##t_1## where ##E_{\uparrow(\downarrow)} = \Pi_{x_{\uparrow(\downarrow)}} + \Pi_{z_{\uparrow(\downarrow)}}##. I.e. the Bomb performs a spin measurement on an undetermined axis x or z. I can be certain that no bombs connected to any member of the ensemble characterised by the pre- and post-selection ##\rho_i,\rho_f## exploded.

I don't see how these inferences are "not permitted by more typical prediction scenarios". All you are saying is that, if the Bomb only goes off if the spin measurement at ##t_1## (which could be either spin-x or spin-z) gives the result "down", then we know that any member of the ensemble that was measured to be spin-x up at ##t_0## or spin-z up at ##t_2## didn't have the bomb explode. What "prediction scenario" would disallow me from drawing this inference?

 

[Morbert]

In classical probability theory ##E^{j_{i}}_{i}## is just some subset of the probability space ##\Omega## and thus ##\lnot E^{j_{i}}_{i}## is just defined as its complement.

In quantum probability however ##E^{j_{i}}_{i}## is represented by some projector or povm, but there is no unique definition for ##\lnot E^{j_{i}}_{i}##. Taking a one-dimensional projector on a ##d##-dimensional Hilbert space for simplicity, ##\lnot E^{j_{i}}_{i}## could any collection of ##d-1## one dimensional projectors onto orthogonal lines. These different ways of representing ##\lnot E^{j_{i}}_{i}## do not give the same probability in the denominator above.

Yeah QM calls for extra diligence. Taking your projective example above, we would have to be careful to ensure ##E^{j_{i}}_{i}## and ##\lnot E^{j_{i}}_{i}## sum to the identity on that measurement's Hilbert space. I.e. ##\lnot E^{j_{i}}_{i} = I - E^{j_{i}}_{i}##. Additionally, when we compute quantities like

$$p(\alpha;\rho_i,\rho_f) = \mathrm{Tr}\left[\rho_fC^\dagger_\alpha\rho_iC_\alpha\right]$$

we have to make sure that ##\mathrm{Tr}\left[\rho_fC^\dagger_{\alpha'}\rho_iC_\alpha\right]=0## for ##\alpha'\neq\alpha## or our probabilities will fail to obey standard probability calculus.

 

[Morbert]

I don't see how these inferences are "not permitted by more typical prediction scenarios". All you are saying is that, if the Bomb only goes off if the spin measurement at ##t_1## (which could be either spin-x or spin-z) gives the result "down", then we know that any member of the ensemble that was measured to be spin-x up at ##t_0## or spin-z up at ##t_2## didn't have the bomb explode. What "prediction scenario" would disallow me from drawing this inference?

Say the bomb instead makes the POVM at some time after post-selection ##\rho_f##. You can no longer be certain that the Bomb will not explode, since a measurement along the x-axis might produce an outcome ##\downarrow##.

 

[PeterDonis]

Say the bomb instead makes the POVM at some time after post-selection ##\rho_f##.

Which means what in plain English? Are you saying the measurement that determines whether the bomb will explode is now made after ##t_2##, instead of at ##t_1## (i.e., before ##t_2##)?

You can no longer be certain that the Bomb will not explode, since a measurement along the x-axis might produce an outcome ##\downarrow##.

Assuming the answer to my question above is yes, then of course changing the time at which the measurement to determine whether the bomb explodes is made will change the probabilities. So what? What is this supposed to prove?

 

[Morbert]

Assuming the answer to my question above is yes, then of course changing the time at which the measurement to determine whether the bomb explodes is made will change the probabilities. So what? What is this supposed to prove?

If the determining measurement was carried out at ##t_1##, the probability that the bomb has not exploded is 1. If the determining measurement will be carried out after ##t_2##, the probability can't ever be 1 no matter what characteristics are used to select the ensemble.

I.e. It's not just that the probabilities change. It's that when an ensemble is selected based on initial and final characteristics, we can identify certainties in outcomes of intervening measurements that are not possible with a more conventional prescription (preparation based on some characteristic in the past, and measurement to be made in the future)

 

[Morbert]

PS Here is a paper discussing this more general ensemble selection: https://journals.aps.org/pr/abstract/10.1103/PhysRev.134.B1410

 

[PeterDonis]

It's that when an ensemble is selected based on initial and final characteristics, we can identify certainties in outcomes of intervening measurements that are not possible with a more conventional prescription

In other words, you're taking post-selection, which is nothing new, calling it "retrodiction", and then trying to claim that it's some kind of new discovery. Obviously if you put more constraints on the ensemble, you are going to be able to draw more inferences. I just don't see that this is any kind of mystery or new discovery.

 

[Morbert]

In other words, you're taking post-selection, which is nothing new, calling it "retrodiction", and then trying to claim that it's some kind of new discovery. Obviously if you put more constraints on the ensemble, you are going to be able to draw more inferences. I just don't see that this is any kind of mystery or new discovery.

I presented a textbook time-symmetric formulation of measurement, more general than conventional prescription like e.g. the one in the stickied thread in this forum, which embeds time asymmetry in its rules. In a conventional experimental ensemble context, this formulation is understood in terms of pre- and post-selection with intervening measurements. But it can also be extended to other contexts. In quantum cosmology, replacing pre-selection with initial conditions, and post-selection with present data. We can see how quantum cosmologists might make inferences about the distant past ( https://arxiv.org/pdf/gr-qc/9712001.pdf ). And how classical retrodiction of the past based on present data alone (as alluded to by CelHolo) arises as a limiting case of quantum retrodiction.

I did not present this as a new discovery. No offense to physicsforums but I prefer to present novel research in peer reviewed journals or conferences. I only presented it as distinct from the typical prescription found in these forums.

 

[PeterDonis]

I presented a textbook time-symmetric formulation of measurement

It's only "time symmetric" because you have added an additional condition to the ensemble, that you know the measurement and its result at the end of some time period of interest as well as at the beginning. The fact that the usual presentation of QM, as in the "7 Basic Rules" sticky that you reference, does not do this is because in most actual cases of interest, we do not have the necessary information (we don't know any measurement result at the end of the time period of interest because that time period has not happened yet). In cases where post-selection is done, we do have the information, and in those cases, what you are describing is simply post-selection.

it can also be extended to other contexts. In quantum cosmology, replacing pre-selection with initial conditions, and post-selection with present data

I'll take a look at the Hartle paper you reference.

I did not present this as a new discovery.

You said (as I quoted before):

we can still make inferences not permitted by more typical prediction scenarios.

But up until your post #12, you didn't describe any new "prediction scenarios" that allow you to make inferences "not permitted by more typical" ones; you just described a restricted ensemble where post-selection has been done in addition to pre-selection based on a known initial state. Calling that "retrodiction" is just changing the words without changing anything substantive.

In post #12, at least, you gave an example where there is a difference from the standard laboratory experiment: in quantum cosmology, we don't know what the initial conditions were, we only know the present data, i.e., we have the "post-selection" data but not the "pre-selection" data. So we are indeed making inferences that are different in nature from the inferences we make in a scenario where we know the initial conditions (because we have explicitly prepared a quantum system in a known state, or because we are observing a natural phenomenon, like the emission of light from distant objects, which is the same as something we have studied in the laboratory so we can assign a known initial condition). If you had given a reference like that at the outset it would have been much clearer why you were bringing this topic up in the first place.

 

[Morbert]

I'm not sure there is any major point of disagreement that warrants continuing the convo so I will leave it here. The only thing I will add is:

The fact that the usual presentation of QM, as in the "7 Basic Rules" sticky that you reference, does not do this is because in most actual cases of interest, we do not have the necessary information (we don't know any measurement result at the end of the time period of interest because that time period has not happened yet.

The paper I linked to in post 10 brings up this interesting point:
"As a matter of fact, in experimental physics selections are frequently based on combinations of initial and final characteristics. Consider a beam of particles that enters a cloud chamber or similar device controlled by a master pulse. For the device to select an event as belonging to a sample to be evaluated statistically, the particle must enter the chamber and, prior to the onset of any manipulation by magnetic fields, etc., satisfy certain requirements. But in order to be counted the particle must also activate the circuits of counters
placed below the chamber; thus, we make the selection on the basis of both the initial and the final state. In
some experiments even intermediate specifications may be imposed in addition to initial and final conditions.
Thus, our formal treatment of initial and final states on an equivalent footing is not inconsistent with experimental procedures used in some investigations."

 

[PeterDonis]

The paper I linked to in post 10 brings up this interesting point

It's worth noting that the kind of "post-selection" described in that quote is due to the inefficiency of our detectors--when we run an experiment, we typically only detect a fraction of the particles produced. The particles we don't detect provide no data so obviously we can't make use of any data from them. It is a valid point that in order to make use of the data from the particles we do detect in such an experiment, we do have to define the ensemble using the final state corresponding to that detection, as well as the initial state corresponding to however we prepared the particles.

However, the kind of "final state" referred to in that quote is not one we are using to distinguish between different possible experimental outcomes for individual particles. It's just some kind of particle counter that verifies that the particle made it all the way through the experiment, or that counts how many particles came out. So the final state only post-selects one ensemble.

In an experiment like Stern-Gerlach, by contrast, there is no single final state we use to post-select an ensemble. The whole point is that we have two possible final states and we want to evaluate the relative frequency of each in order to compare with predictions. This is the kind of scenario a formulation like the "7 Basic Rules" is intended to describe, since it's the kind that is usually presented in introductory textbooks. We have had discussions in other threads about the fact that this formulation is limited--there are more general classes of experiments that it does not cover, at least not as it is presented in the "7 Basic Rules" thread.