Gradient of a complex expression

[dienchu]

Hello,
My question is concerning how to compute the complex gradient of the following cost functional with respect to W:

F=Ʃ_i=1:M ||y_i-Go*W_i||^2 + Ʃ_i=1:M ||W_i - X*(E_i - Gc*W_i)||^2

Where the summations go from i=1 to i=M and the dimensions of the diferent elements are:
y: Nx1
Go: Nxn
W: nx1
X:nxn
E_i:nx1
Gc:nxn
And ||..||^ 2 indicates the square norm.
The elements of the different matrix and vectors are complex numbers

My problem is that I need to compute the gradient to apply a conjugate gradient minimization algorithm to minimize the cost functional. Any help or reference about how to compute gradients of this type of expressions (vectors and matrix) will be appreciated.
Thank you very much.

 

[jvicens]

Hello,

Thank you for your question. Computing the complex gradient of a cost functional can be a challenging task, but there are some general steps that can help guide you in the process.

First, it is important to understand the basic principles of gradient computation. The gradient of a function is a vector that points in the direction of greatest increase of the function. In other words, it tells us the direction in which the function is increasing the fastest. The gradient is computed by taking the partial derivative of the function with respect to each variable.

In your case, the cost functional F is a sum of two terms. Let's start by computing the gradient of the first term:

Ʃ_i=1:M ||y_i-Go*W_i||^2

To compute the gradient, we need to take the partial derivative of this term with respect to each variable, which in this case are the elements of W. Since the expression is a sum, we can compute the gradient of each term separately and then add them together.

Let's take a closer look at the first term, ||y_i-Go*W_i||^2. This can be expanded as:

(y_i-Go*W_i)^H(y_i-Go*W_i)

where the superscript H denotes the complex conjugate transpose. Using the chain rule, we can compute the gradient of this term with respect to W_i as:

-2Go^H(y_i-Go*W_i)

Similarly, the gradient of the second term can be computed as:

-2X^H(E_i-Gc*W_i)

Now, since the cost functional F is a sum of these two terms, the gradient of F with respect to W_i is simply the sum of these two gradients:

-2Go^H(y_i-Go*W_i) - 2X^H(E_i-Gc*W_i)

Once you have computed the gradient for each element of W, you can use this information to update the values of W using a conjugate gradient minimization algorithm.

It is worth noting that the complex nature of the elements in your expressions may introduce some additional complexity in the gradient computation. However, the general principle remains the same. I suggest referring to some resources on complex gradient computation, such as this paper by Nocedal and Wright, for more in-depth information and examples.

I hope this helps. Best of luck with your research!