Fourier Transform: Geometric Interpretation?

[waves and change]

Hi, outside the mathematical proof that shows that sines of different frequency are orthogonal... is there geometric interpretation/picture of this phenomena?

 

[Stephen Tashi]

Suppose we want to represent a function f(x) as a linear combination of the sum of three functions a(x),b(x),c(x). There are 3 possibilities

1) f(x) cannot be represented as a linear combination of a(x), b(x), c(x).
2) f(x) can be represented as a linear combination of a(x), b(x), c(x) in more than one way
3) f(x) can be represented as a linear combination of a(x), b(x), c(x) in exactly one way.

It's fair to say that these possibilities depend on the shapes of the graphs of f(x), a(x), b(x), c(x). So there is a geometric aspect to situation. I don't think there is a simplistic way to visualize this geometric aspect.

The advantage of using orthogonal functions a(x), b(x), c(x) is that if it comes down to a choice between case 2) and case 3), we get get case 3). If we don't have orthogonal functions, we might have a situation such as
5a(x) +4b(x) = c(x). Then if we have f(x) = 5a(x) + 4b(x) + 10 c(x), we also have f(x) = 11 c(x), getting two different representations of f(x).

The general idea is that a function cannot be expressed as a linear combination of other functions that are orthogonal to it. So (intuitively) if we express a function f(x) as a linear combination of functions that are orthogonal to each other, we don't have to worry about getting two different representations of f(x).

If a(x) = sin(2x), you can "see" that a(x) cannot be represented as a linear combination of sin(1x) and sin(3x) because the periods won't work out. Perhaps the best we can do as far as seeing a set of functions are mutually independent is to perceive some reason why each one cannot be written as a linear combination of the others. "Orthogonal" is, in a manner of speaking, an extreme case of independence. One might think of orthogonal functions as being "uncorrelated".

 

[sophiecentaur]

is there geometric interpretation/picture of this phenomena?

Here is a 'geometrical' and pictorial argument to support the mathematical one. Plot two equal amplitude sinusoids along the same axis (from - infinity to + infinity. (If they have periods of n and m where n and m are rational and different, you can limit the range to the repeat length of the 'beat'). Take any point along the axis and examine the two values (amplitudes) and there will always be another point, somewhere along the axis, with the same two amplitudes but a difference in sign. So the the sum of the products of that pair values over the total interval will be zero. The sum of all those produces will be zero. If n=m, the sinusoids are always in step and there is no cancellation.
This is just an arm waving description of the cross correlation between the two functions.

 

[waves and change]

Thank you for the insight, when you say somewhere along the x-axis there will be a point with a value with opposite sign. Is there a way to describe quantitively where that point is given the initial point chosen ?

 

[waves and change]

Suppose we want to represent a function f(x) as a linear combination of the sum of three functions a(x),b(x),c(x). There are 3 possibilities

1) f(x) cannot be represented as a linear combination of a(x), b(x), c(x).
2) f(x) can be represented as a linear combination of a(x), b(x), c(x) in more than one way
3) f(x) can be represented as a linear combination of a(x), b(x), c(x) in exactly one way.

It's fair to say that these possibilities depend on the shapes of the graphs of f(x), a(x), b(x), c(x). So there is a geometric aspect to situation. I don't think there is a simplistic way to visualize this geometric aspect.

The advantage of using orthogonal functions a(x), b(x), c(x) is that if it comes down to a choice between case 2) and case 3), we get get case 3). If we don't have orthogonal functions, we might have a situation such as
5a(x) +4b(x) = c(x). Then if we have f(x) = 5a(x) + 4b(x) + 10 c(x), we also have f(x) = 11 c(x), getting two different representations of f(x).

The general idea is that a function cannot be expressed as a linear combination of other functions that are orthogonal to it. So (intuitively) if we express a function f(x) as a linear combination of functions that are orthogonal to each other, we don't have to worry about getting two different representations of f(x).

If a(x) = sin(2x), you can "see" that a(x) cannot be represented as a linear combination of sin(1x) and sin(3x) because the periods won't work out. Perhaps the best we can do as far as seeing a set of functions are mutually independent is to perceive some reason why each one cannot be written as a linear combination of the others. "Orthogonal" is, in a manner of speaking, an extreme case of independence. One might think of orthogonal functions as being "uncorrelated".

Thanks for the insight but nonetheless thinking of functions as being uncorrelated doesn’t really do justice geometrically. You can easily picture 2 vectors being uncorrelated by having a 90 degree angle between them. I’m trying to imagine an analogue with regards to sine functions ...

Thank you

 

[Stephen Tashi]

You can easily picture 2 vectors being uncorrelated by having a 90 degree angle between them..

If you can picture that for cases like a pair of thousand-dimensional vectors, then you're in business!

I’m trying to imagine an analogue with regards to sine functions ...

Just imagine the sine functions approximated by data at 1000 points.

Perhaps what you mean is that we can easily visualize a pair of 2 or 3 dimensional vectors as being "uncorrelated" when there is a 90 degree angle between them (?).

 

[waves and change]

If you can picture that for cases like a pair of thousand-dimensional vectors, then you're in business!
Just imagine the sine functions approximated by data at 1000 points.

Perhaps what you mean is that we can easily visualize a pair of 2 or 3 dimensional vectors as being "uncorrelated" when there is a 90 degree angle between them (?).

You used the word uncorrelated I’m trying to see if there is any geometric insight here. Imagining a sine function at 1000 points ? Don’t see what information that conveys ?

 

[sophiecentaur]

Thank you for the insight, when you say somewhere along the x-axis there will be a point with a value with opposite sign. Is there a way to describe quantitively where that point is given the initial point chosen ?

You wanted a geometric description. Personally I would rather stick with the Maths - which can easily be proved - and that is the "quantitative" way. My answer would have to be that, for a graphical demonstration, you could take two sinusoids (sketched) and, by inspection (as they say) you can pick a point on the x-axis and scan across to find another point that will satisfy the requirement of same amplitudes and opposite sign. Pick f and 2f for a start and the picture shows what's going on clearly. It's not rigorous but simple geometrical proofs seldom are. I'm not sure why you feel the need to go to deeply into the geometrical way when the standard textbook analytical approach to deriving the FT does it all.

 

[waves and change]

You wanted a geometric description. Personally I would rather stick with the Maths - which can easily be proved - and that is the "quantitative" way. My answer would have to be that, for a graphical demonstration, you could take two sinusoids (sketched) and, by inspection (as they say) you can pick a point on the x-axis and scan across to find another point that will satisfy the requirement of same amplitudes and opposite sign. Pick f and 2f for a start and the picture shows what's going on clearly. It's not rigorous but simple geometrical proofs seldom are. I'm not sure why you feel the need to go to deeply into the geometrical way when the standard textbook analytical approach to deriving the FT does it all.

Because I don’t just read a textbook see a proof and robotically memorize it. Geometric interpretations can give deep insight when you “ take a step back “ in my opinion. Thank you for the “ geometric insight “ any sources you know of that give insight into this?

 

[sophiecentaur]

Because I don’t just read a textbook see a proof and robotically memorize it.

I agree that robotically memorising a proof is totally pointless. I suspect this is a cop out and that you don't actually want to get to grips with this. You will never have a good grasp of Physics is you are not prepared to address it in the language that is best suited to it (Maths). I am sure that you use Maths at some level and that you actually find it useful. It is unlikely that you use a pictorial method for working out your personal budget and that you use Arithmetic or even algebraic equations for much of Science.
Calculus and the transforms that it allows you to perform will give a depth of understanding that pictures never will.

 

[waves and change]

I agree that robotically memorising a proof is totally pointless. I suspect this is a cop out and that you don't actually want to get to grips with this. You will never have a good grasp of Physics is you are not prepared to address it in the language that is best suited to it (Maths). I am sure that you use Maths at some level and that you actually find it useful. It is unlikely that you use a pictorial method for working out your personal budget and that you use Arithmetic or even algebraic equations for much of Science.
Calculus and the transforms that it allows you to perform will give a depth of understanding that pictures never will.

Who said I don’t want to address phenomena mathematically. You don’t know anything about me yet want to make these blanket sarcastic statements about my budget. I love math and studied math and physics in college. I’m wasting my time with someone not worth my time. I found a few good sources on the geometric interpretations of Fourier analysis.

Thanks anyway

 

[sophiecentaur]

Who said I don’t want to address phenomena mathematically. You don’t know anything about me yet want to make these blanket sarcastic statements about my budget. I love math and studied math and physics in college. I’m wasting my time with someone not worth my time. I found a few good sources on the geometric interpretations of Fourier analysis.

Thanks anyway

No need to take offence. The fact that you talked in terms of learning like a robot implied to me that you didn't want to appreciate the beauty of the way transforms work. You wrote as if it held no magic for you.

Who said I don’t want to address phenomena mathematically.

You did, in your first post.

 

[waves and change]

I said I wanted insight into the phenomena geometrically because I have learned the pure mathematics proof. That’s why the question states geometric intuition. I would assume you understand that statement hopefully.

 

[waves and change]

No need to take offence. The fact that you talked in terms of learning like a robot implied to me that you didn't want to appreciate the beauty of the way transforms work. You wrote as if it held no magic for you.

You did, in your first post.

Oh said in my post that I don’t want to address phenomena mathematically? Haha I guess English isn’t your forte ... I said simply that I was looking for a geometrical interpretation of a specific topic.. Fourier transform. If you can’t differentiate between those two statements... you have a problem ...

 

[sophiecentaur]

Oh said in my post that I don’t want to address phenomena mathematically? Haha I guess English isn’t your forte ... I said simply that I was looking for a geometrical interpretation of a specific topic.. Fourier transform. If you can’t differentiate between those two statements... you have a problem ...

If you really are into the Maths then why did you talk about memorising a proof like a robot? I don't know of any robots with an insight into Maths (yet). I'm sorry but I can only respond to what your write.
Perhaps we should stop bickering now. I am glad you have found what you wanted.

 

[waves and change]

So you got emotional over the memorizing proofs robotically...I see... sorry to offend you. Truth is many proofs are so complex that even professional physicists need to simply memorize them ( if they have any use for them etc). But no worries agree to disagree

 

[Stephen Tashi]

You used the word uncorrelated I’m trying to see if there is any geometric insight here. Imagining a sine function at 1000 points ? Don’t see what information that conveys ?

That confirms what I suspected. When youj said it was easy to tell if two vectors were uncorrelated, you meant vectors in 2 or 3 dimensions.

You are probably familar with doing linear regression. For example if X = height and Y = weight and we have such paired data from a sample of people, we can ask what the "best" way to approximate Y as AX + B, meaning what are the values of A and B that minimize the square error ##\sum (Y_i - AX_i - B)^2 ## If the best value of A = 0 then X and Y are uncorrelated.

Consider the two 3-dimensional vectors X = (1,4,2) and Y = (6,-1,-1). Treat them as paired data (1,6), (4,-1), (2,-1). If you plot these points perhaps you can have some geometric intuition that the best value of A is A = 0. (i.e a horizontal regression line does better than a sloped one).

I haven't tried to do a similar thing for 1000 points of two functions. It might be revealing. For example for ##f(x_i) = sin(x_i)## and ##g(x) = sin(2 x_i)##, what does a plot of the data ##(f(x_i), g(x_i))## look like for evenly spaced ##x_i## in ##[0,2\pi]##?

 

[waves and change]

Thank you! I will attempt this later on!

 

[mfb]

@Stephen Tashi: What you describe in post 2 just needs linear independence. Being orthogonal is a stronger condition and you can always find a metric where they are not orthogonal.

It makes sense to have them orthogonal here - it is the most symmetric case.

 

[Svein]

Because I don’t just read a textbook see a proof and robotically memorize it.

If you are up for a final math exam (at the master level) you better have memorized the main point the proof. The proof of several theorems have been painstakingly worked out by exceptional mathematicians over the course of several years. I do not think that you will be able to come up with your own proof at the spur of the moment standing before an examination board.

 

[osilmag]

Hi, outside the mathematical proof that shows that sines of different frequency are orthogonal... is there geometric interpretation/picture of this phenomena?

This picture might help show how the signal amplitude is plotted against frequency and time. This is from a Google Image search.

 

[waves and change]

If you are up for a final math exam (at the master level) you better have memorized the main point the proof. The proof of several theorems have been painstakingly worked out by exceptional mathematicians over the course of several years. I do not think that you will be able to come up with your own proof at the spur of the moment standing before an examination board.

What relevance this post had? ...None

 

[waves and change]

That confirms what I suspected. When youj said it was easy to tell if two vectors were uncorrelated, you meant vectors in 2 or 3 dimensions.

You are probably familar with doing linear regression. For example if X = height and Y = weight and we have such paired data from a sample of people, we can ask what the "best" way to approximate Y as AX + B, meaning what are the values of A and B that minimize the square error ##\sum (Y_i - AX_i - B)^2 ## If the best value of A = 0 then X and Y are uncorrelated.

Consider the two 3-dimensional vectors X = (1,4,2) and Y = (6,-1,-1). Treat them as paired data (1,6), (4,-1), (2,-1). If you plot these points perhaps you can have some geometric intuition that the best value of A is A = 0. (i.e a horizontal regression line does better than a sloped one).

I haven't tried to do a similar thing for 1000 points of two functions. It might be revealing. For example for ##f(x_i) = sin(x_i)## and ##g(x) = sin(2 x_i)##, what does a plot of the data ##(f(x_i), g(x_i))## look like for evenly spaced ##x_i## in ##[0,2\pi]##?

Thank you for the insight in your posts which steered me in the right direction and I was able to get a grasp of the geometrical aspect after finding a few good online papers. Thanks again!

This thread can be closed