[itex]L^2(-\infty, +\infty)[/itex] and LTI systems

[solanojedi]

Hi everyone,
I'm an electrical engineer trying to make a little bit of order in my mind about eigenfunction theory, sturm liouville operators (regular and singular) on my own free time and I thought I was grasping almost everything, but I had a little doubt that is actually coming from the field of control theory.

First, I start from the math: I learned (I'll be for sure approximate, I apologize, but correct me if I'm wrong) that in Hilbert spaces it is always possible to define a basis made of eigenfunctions and every operator and function existing inside the space can be rewritten using that basis. However, how to find this basis?
There are particular operators called Sturm Liouville operators that, defining an eigenvalue problem, can generate a set of functions and those functions are the basis for the corresponding space I'm working with. In particular, there are Sturm Liouville "singular" problems that are defined over a space that is not "finite" but extends from [itex]-\infty[/itex] to [itex] +\infty [/itex]: one of them, and for an engineer the most important, is [itex] L^2(-\infty, +\infty) [/itex].
In this space, the operator [itex]\frac{d^2}{dx^2}[/itex] defines an Sturm Liouville singular problem and the solution of this eigenvalue problem is an infinite set of complex exponentials. From this theory we get the Fourier transform theory.
Ok, this was a sort of a very general summary of what I have in my mind.

Now, switching to control theory: we say that for LTI systems (that are described with LTI operators) the complex exponential is an eigenfunction of the operator and hence I can use the Fourier transform to analyze those systems.
This, in my opinion, is only because I'm dealing with signals and operators that by definition are [itex] L^2(-\infty, +\infty) [/itex]. Hence, I believe LTI operators are only a special case of the general operators that are in [itex] L^2(-\infty, +\infty) [/itex].

Here's my question: what happens if the system is not LTI but for example linear time variant? What kind of signals (and operators) I'm dealing with? I guess that they are not anymore [itex] L^2(-\infty, +\infty) [/itex], and so I cannot use the same instruments like Fourier/Laplace transform , but this is just a guess.

Thank you in advance!
Nicola

 

[mathman]

[tex]L^2(-\infty, \infty )[/tex] has nothing to do with time variant. Physical interpretation would be the issue.

 

[solanojedi]

Hi mathman,
thank you for your answer but I don't understand. LTI systems must "live" inside L2, otherwise I won't be able to use tools like the Fourier transform.
My questions were: 1) LTI systems are a special case of L2? (hence it's not their peculiarity to have complex exponentials as eigenfunctions, since the whole space is equipped with this basis) 2) what happens if the system is not LTI? I probably cannot use the transforms, am I right?

Thank you again,
Nicola

 

[mathman]

It is hard for me to respond, since (as a mathematician) I am not familiar with LTI systems. I would need a precise definition.

 

[solanojedi]

I'm not really able to provide a precise (in a mathematical sense) definition. Generally speaking, LTI systems are mathematical models that involve operators that are continuos, linear and invariant to shifts (in this case, time-shifts).
A more extensive discussion can be found here: https://en.wikipedia.org/wiki/Linear_time-invariant_theory , where there is also a "proof" that complex exponentials are eigenfunctions of these operators.

 

[mathman]

I looked at your reference and I will have to beg off further analysis (at my age I find it difficult to absorb new material). Mathematically a function is a member of [tex]L_2(-\infty ,\infty ) [/tex] if its square is integrable over the domain. One important property is that you can take Fourier transforms.
In the distant past I did work with what were called stationary Gaussian process. The main concern was about their autocorrelation functions., with Fourier transforms being their power spectra. This could have some relevance to what you are looking at.

 

[jasonRF]

Here's my question: what happens if the system is not LTI but for example linear time variant? What kind of signals (and operators) I'm dealing with? I guess that they are not anymore [itex] L^2(-\infty, +\infty) [/itex], and so I cannot use the same instruments like Fourier/Laplace transform , but this is just a guess.

Linear operators that are time variant would be, for example, ordinary differential equations with variable coefficients. Some of these are singular and some are regular. Think of a linear filter made up of resistors, capacitors and inductors that includes a variable capacitor that is connected to a knob that a user can control. As the user turns the knob the system is changing. The ordinary differential equation describing the output of the filter would have a coefficient that is time-varying. I am not a mathematician, but I would think that whether an [itex] L^2(-\infty, +\infty) [/itex] input produces an [itex] L^2(-\infty, +\infty) [/itex] output will depend upon the particular system.

You are correct in that using Fourier or Laplace transforms to analyze a differential equation with variable coefficients is often not so useful (although sometimes they are very useful, such as analyzing Airy's equation). Since you are learning about Sturm-Liouville theory, I am assuming that you know something about Green's functions? Once you know the Green's function then you know the response of the system represented by the differential equation for a given input, whether or not it is time-invariant.

By the way, I'm hoping you already know that Laplace transforms are useful for a much larger class of functions than [itex] L^2(-\infty, +\infty) [/itex]. For example, it is easy to find the Laplace transform of [itex]u(t) \, e^t[/itex], (where the step function [itex]u(t)[/itex] is 0 for t<0 and 1 for t>0). That is why Laplace transforms are so useful for analyzing both stable and unstable systems.

Jason

 

[solanojedi]

Hi Jason, thank you very much for your answer.
I believe I found an answer to my doubt in this thread: https://dsp.stackexchange.com/quest...and-transform-only-applicable-for-lti-systems and I'll try to summary the answer.
LTI systems are a special case of linear operators. When I'm dealing with signals that are [itex]L^2(-\infty +\infty)[/itex], these signals can be expressed as integral of complex exponentials and, since LTI operators act from [itex]L^2[/itex] to [itex]L^2[/itex], the complex exponentials are also eigenfunctions of the operator.
The nice thing about these systems is that their Fourier transform is given by a product of the input signal by a complex number (a tranfer function, basically).
If the system is not time invariant or not linear, we won't get a product in the frequency domain, but a convolution, that is much more complicated to handle.

This answers my original question.

You answer, Jason, raise me more doubts :)
I'm not a mathematician either, but I think that these results hold if the system (wheter linear, non linear, time variant...) transforms signals from [itex]L^2[/itex] to [itex]L^2[/itex]. If not, I actually don't know what happens.
I'm not particularly familiar with Green functions (only the general concept - I'm still trying to understand the "big picture"), but from your discussion it seems like they are the generalization of the impulse response (that is defined only in LTI system) to more general systems. Could it be correct?

Fourier transforms and Laplace transforms are usually used to solve ODE and PDE. Laplace was "introduced" because Fourier transform is defined in L2 and in that space esponentials (typical solutions of ODE) are not defined, hence it was necessary something else that was able to handle them. But when I have to use one and when the other in ODE/PDE? Now I think I'm going to search for some reference where the same ODE/PDE is solved with the two different approach. I expect that with Fourier approach I'm going to miss some solution.

I think we are kinda going off topic, but I thank you very much for your replies, it helps me think and make order in my thoughts.

Nicola

 

[jasonRF]

I'm not particularly familiar with Green functions (only the general concept - I'm still trying to understand the "big picture"), but from your discussion it seems like they are the generalization of the impulse response (that is defined only in LTI system) to more general systems. Could it be correct?

Correct

Fourier transforms and Laplace transforms are usually used to solve ODE and PDE. Laplace was "introduced" because Fourier transform is defined in L2 and in that space esponentials (typical solutions of ODE) are not defined, hence it was necessary something else that was able to handle them. But when I have to use one and when the other in ODE/PDE? Now I think I'm going to search for some reference where the same ODE/PDE is solved with the two different approach. I expect that with Fourier approach I'm going to miss some solution.

Yes, the Fourier transform approach will throw out those solutions that are not Fourier transformable.

Jason

 

[solanojedi]

Ok, so from what I understand I can get different solutions of an ODE/PDE depending on the hypothesis of the space I'm working with.
If I suppose to work with functions in [itex]L^2(-\infty, +\infty)[/itex], then the Fourier transform is valid and I can exploit the fact that the unknown function [itex]x(t)[/itex] in a ODE can be written with the eigenfunction basis [itex]e^{i\omega t}[/itex] as the inverse Fourier transform [itex]\int^{+\infty}_{-\infty} X(f) e^{i\omega t} d\omega[/itex]. However, this approach "loses" other possible solutions that do not live in [itex]L^2(-\infty, +\infty)[/itex].

To get all the solutions (like exponentials that are not square integrable) and hence be able to deal also with the unstable solutions described by the same ODE, we need the Laplace transform that can handle exponential growths.

When we apply the Laplace transform to an ODE we basically perform the same procedure as the Fourier transform but the premises are different: we suppose to deal with another "space" (if it is correct to call it space) with more "rich" signals than [itex]L^{2}(-\infty, +\infty)[/itex]. Then we suppose that the usual signal [itex]x(t)[/itex] of the ODE can be written as [itex]\int^{+\infty}_{0} X(s) e^{s t} ds[/itex], as if in this new "space" there was an eigenfunction basis made of exponential with a complex exponential [itex]e^{s t}[/itex].
Does it make sense?

EDIT: a quick question: LTI operators, since they are sums of derivatives, can somehow viewed as combinations of Sturm Liouville operators? In this case it can come from SL theory that complex exponentials are eigenfunctions for LTI operators?