Is Occam's Razor Valid? Understanding Its Rationale

[Scientific Method]

What is the rationale behind the validity of Occam's Razor?

Thanks.

 

[arildno]

Given the hypothetical situation in which two philosophically/mathematically distinct theories provides EXACTLY the same predictions, both theories may be regarded as equally (in-)valid.
Hence, it is perfectly legitimate on purely aesthetic grounds to choose to work with the "simplest" model.

However, such a situation does not that often occur in practice. Competing theories in physics will usually be characterized by slightly different predictions, and hence, Occam's razor cannot be used as the distinguishing principle between them.
Rather, experiments must be undertaken to find out which theory gives the best predictions.

 

[NeutronStar]

What is the rationale behind the validity of Occam's Razor?

Thanks.

I don't believe that there is any formal rationale behind Occam's Razor. It's more of an opinion than anything.

It certainly isn't a formal requirement of "The Scientific Method". I mean, a theory can't be said to in incorrect or invalid simply because it doesn't comply with Occam's Razor.

In other words, proving that a theory doesn't comply with Occam's Razor does not constitute a valid proof that a theory is wrong.

I personally don't buy into Occam's Razor entirely. I do like simplistic theories, but if a theory can offer a comprehensible explanation of phenomena by introducing ideas that may not be testable I have no problem with that. I think that such a theory should at least be considered until some other theory can do better. String Theory is like the perfect example of this.

However, if a more simplistic theory lacks a comprehensible explanation then why should I buy into that either? So I favor explanatory theories over simplistic one if they provide a better understanding of what might be going on ontologically.

I call that "The NeutronStar Hatchet Principle". (i.e. Hack off the theories that don't offer a comprehensible explanation.)

 

[Aquamarine]

In its original form, Occam's razor only says that one should choose the "simpler" theory among those that predict equally well.

"Simpler" is unclear. Another problem it that an immensly complex theory ripe with ad hoc exceptions should be choosen before a very simple theory that give almost as good predictions. It should be noted that it is always possible to make a theory fit all known facts perfectly by adding more and more ad hoc exceptions. For example, if an experiment proves a theory wrong one can always add the ad hoc exceptions that theory always works except during the time and place that the experiment took place. No matter how many experiments one does, there is always an infinite number of theories that fit all the data perfectly.

Science is somewhat similar to curve-fitting. That is, finding an equation that passes through or near some finite number of points. There is an infinite number of equations that fit all the points perfectly.

Modern attempts to define Occam's razor try to balance complexity of the theory and how well the theory predicts.

One attempt might be that one should choose a shorter explanation. With length of explanation defined as the size of the computer program needed to write a formula that predicts the data + all the data that are not predicted by the formula.

And maybe surprisingly, this seems to give useful results.
http://www.csse.monash.edu.au/~lloyd/tildeMML/Intro/

 

[Ivan Seeking]

In its popular usage, Occam's razor is often inappropriately treated as nearly a law of physics. It's a guide but that's all. One common misapplication is to ignore evidence since Occam's razor implies that the evidence must be wrong.

 

[Chronos]

Occam's razor probably creates as much confusion as it dispels. But then again, Occam is not historically credited with saying:
"Entia non sunt multiplicanda praeter necessitatem." or "Entities should not be multiplied more than necessary."

His actual words were:
"Pluralitas non est ponenda sine neccesitate" or "Plurality should not be posited without necessity."

This is good practical advice, as is "Do not multiply entities unnecessarily" and "Of two competing theories or explanations, all other things being equal, the simpler one is to be preferred.", which is another saying he did not say.

While occasionally useful, these principles have no scientific validity. You may as well say a screwdriver is more valid than a pipe wrench because it has fewer moving parts.

 

[Ivan Seeking]

While occasionally useful, these principles have no scientific validity. You may as well say a screwdriver is more valid than a pipe wrench because it has fewer moving parts.

That's memorable! You just made my quotes for life list.

Edit: right after...Eddington once said, proof is the alter on which mathematicians self-flaggelate themselves.

 

[GeD]

I think that some of the peeps who have written about the principle have got it right.

The root of the problem is in the part of the statement is nowadays interpreted as: that the simpler explanation (of two theories with similar success at predicting) is more likely to be correct.

Since the validity will only be shown according to its success of predicting phenomena, neither of those theories will ever be shown to be correct, it is simply a matter of which one is easier to use. The actual ockham's razor did talk about simplicity and ease of use, not about either being more correct - this was a misinterpretation.

 

[Aquamarine]

Occam's razor probably creates as much confusion as it dispels. But then again, Occam never what he is usually attributed having said:
"Entia non sunt multiplicanda praeter necessitatem." or "Entities should not be multiplied more than necessary."

His actual words were:
"Pluralitas non est ponenda sine neccesitate" or "Plurality should not be posited without necessity."

This is good practical advise, as is "Do not multiply entities unnecessarily" and "Of two competing theories or explanations, all other things being equal, the simpler one is to be preferred.", which is another saying he did not say.

While occasionally useful, these principles have no scientific validity. You may as well say a screwdriver is more valid than a pipe wrench because it has fewer moving parts.

Even the old versions of Occam's razor are used in science:
http://en.wikipedia.org/wiki/Occam's_Razor

Newer versions have much wider application. I would be careful before dismissing a whole division of information theory. And this requires an alternative explanation of how science chooses among the infinite number of theories that fit all known data. And an alternative explanation why an theory with many ad hoc exceptions is bad.

Stanford encyclopedia on simplicity:
http://plato.stanford.edu/entries/simplicity/

Chemistry:
http://www.hyle.org/journal/issues/3/hoffman.htm

Molecular biology:
http://www.csse.monash.edu.au/~lloyd/tildeStrings/

 

[Aquamarine]

Even physics:
http://citeseer.ist.psu.edu/fox98we.html

 

[Icebreaker]

My apologies for the threadjacking: Aquamarine, are you the same one from the ASC forums?

 

[Aquamarine]

My apologies for the threadjacking: Aquamarine, are you the same one from the ASC forums?

No. Obviously a person with good taste.

 

[Rainer]

Chronos has a valid point. Ockham was a radical nominalist--and according to more Enlightenment period terms, an empiricist. Essentially, Ockham was against any sort of induction. Ockham was, ultimately, anti-sceintific--he held that the best we can accomplish is the recording of patterns in nature without inducing any sort of generality or the apprehension of any univeral natural law (like Newton's laws.)

As for simple explanations as used scientifically, this twists Ockham's thought towards the opposition of his original intention. Theories ought to have a sort of simplicity (whatever the **** that means) but what is intended in such a statement is that there should not be "ad hoc" theories with additions for every idiosyncratic event--that to achieve generality, you must be as abstract (inductive, "nonempirical"; the term doesn't *really* exist, I use it for convenience) as possible.

 

[AKG]

It seems, after looking at that Wikipedia article and looking at what Occam really wrote, that his razor states that if two theories have the same explanatory power, but one makes more assumptions than the other, then the better one is the one that makes fewer assumptions. Prima facie, there is nothing confusing or aesthetic about this, it gives a quantifiable way to compare two theories, and assuming all assumptions have an equal "chance" of being wrong, it makes perfect sense to choose the one that makes fewer assumptions. Of course, some assumptions are more "reasonable" than others, but this is perhaps more contingent on subjective judgment, and so either Occam is stating that we ought to choose a theory that makes one whacky assumption over one that makes two "reasonable" ones (which is terrible advice), or he's saying that we choose the one that, in total, is more reasonable and presumes less in its assumption, which really is a subjective judgement, which is something we would do anyways (so it's rather empty advice). There is no way to quantify how "likely" an assumption is to be right, we just have a feeling for which ones seem like they will be right, and which ones will not, so it is something like an aesthetic judgment.

So, is Occam's Razor valid? Well, giving him the benefit of the doubt it says that we ought to choose the theory that makes the most reasonable-seeming assumptions. It follows that Occam's Razor seems reasonable. Is it valid, i.e. will it always give us the right theory? No, because the most reasonable-seeming assumptions don't always turn out to be true. So Occam's advice, (giving him the beneficial interpretation) is good advice, but not immaculate. Whether advice needs to be immaculate to be valid, or simply good, is also a subjective matter, but a trivial one and open to any interpretation you want to make of it.

A related question: why is it called a Razor? And are there other Razor's, or just Occam's?

 

[Scientific Method]

These explanations have been very helpful, thank you. So basically, why say (4/8) when it is simpler and just as valid to say (1/2). So, holding the view that efficiency is good, why choose an explanation that is a multiple of a simpler yet equally valid explanation.

 

[Aquamarine]

These explanations have been very helpful, thank you. So basically, why say (4/8) when it is simpler and just as valid to say (1/2). So, holding the view that efficiency is good, why choose an explanation that is a multiple of a simpler yet equally valid explanation.

I think you are right that one assumption is that theories should be useful or efficient. And that Occam's razor is one rule for finding such theories.

Now many might say that 4/8 is that same as 1/2 and that Occam do not apply in this case. That the different versions of Occam's razor is a rule for choosing between different theories. They would consider 1/2 and 333,333,333/666,666,666 to be equally simple.

The more modern version of Occam's razor I mentioned in my first post would actually choose 1/2 before 4/8. It takes a longer binary code to write 4 / 8 than 1 / 2. And the modern rule says that the shorter code is preferable.

In a computer a shorter code might be preferable because consuming less memory and allowing greater calculation speed. Generalizing, if humans are trying to model the world then a theory that takes less place to store and allows faster calculations might be preferable.

 

[HallsofIvy]

I've said this before but apparently not in this thread so I'll say it again!

A key point in science is that you can't prove an hypothesis: science advances by disproving alternate hypotheses, narrowing the field of those that might be true.

The simpler hypotheses are the ones that can be disproved faster so it is more efficient to start with them- it's not that simpler hypotheses are more likely to be "true", just that it will be easier to disprove them!

 

[Nereid]

These explanations have been very helpful, thank you. So basically, why say (4/8) when it is simpler and just as valid to say (1/2). So, holding the view that efficiency is good, why choose an explanation that is a multiple of a simpler yet equally valid explanation.

Not to throw cold water on this line of thinking, but in what sense is (4/8) 'simpler' than (1/2)? There are, of course, obvious answers, but if you dig deeper, aren't they just as 'theory-laden' as the explanations themselves? In what sense is something an 'equally valid explanation'? Is not one woman's 'efficiency' another man's 'conflation/confusion of assumptions'?

 

[Chronos]

Information theory. Fewer bits are required to code 1/2.

 

[Nereid]

Information theory. Fewer bits are required to code 1/2.

Indeed; so if all we have is numbers, information theory gives us a handle on how to make them as simple as possible? But what if the numbers are (1/1), (4/8), (27/81), ...? And how do we apply information theory to GR (as a theory)?

 

[Aquamarine]

A theory is a better than the data it attempts to explain (describe) if the description is shorter than the data but describes all the data. A theory is better than another theory if it is shorter but describes the data equally well.

In essence, better compression of the data is better science. According to one modern version of Occham's razzor.

 

[PhilipF]

'Occam' + 'Ockham '+ 'Occham' !
"immo entia non sunt triplicanda praeter necessitatem"

 

[NateTG]

Occam's Razor as a Pragmatic Principle

Really, one of the strongest arguments in favor of Occam's razors is that they are pragmatic - that is, using them makes things easier.

For example:
Choosing between two theories that make identical predictions, but where one has additional assumptions. Clearly, the theory with only a subset of the assumptions will be valid in at least all of the situations that the one that has more applies to, and may be applicable in other situations as well.
A famous example is the notions of planar and non-planar geometry and Euclid.

Or:
Choosing between two theories that make the same predictions, but one involves massive computation, and the other one does not. Compare, for example, object colisions of human-scale objects at human-scale speeds in Einsteinian, quantum and Newtonian mechanics. Clearly Newton is the pragmatic choice unless you're dealing with *lots* of sig figs.

The reason that Occam's razor is difficult is that simple is often a subjective choice. This is interesting in light of, for example, quantum mechanics interpretations where there are a number of choices - Many Worlds, Bohmian Mechanics, Von Neuman's 'just calculate', or the Copenhagen interpretation, and a number of esoteric theories can all be shown to make identical predictions, but have rather different intellectual and philosophical issues associated with them. Determinism, locality, and realism - which seems to be a category of 'nice' properties - can't coexist according to current experimental results, but there's no clear consensus about what to throw out.

 

[Chronos]

... And how do we apply information theory to GR (as a theory)?

An interesting question. While a bit off topic, I can't resist tossing you a couple bones :

Quantum information and general relativity
http://arxiv.org/abs/quant-ph/0405127

General Relativity and Quantum Cosmology
http://arxiv.org/abs/gr-qc/0501076

 

[Hugo Holbling]

http://www.galilean-library.org/or.html on Ockham's Razor might be of interest.