Understanding the Cost Function in Machine Learning: A Practical Guide

[emmasaunders12]

Could someone please help me work through the differentiation in a paper (not homework), I am having trouble finding out how they came up with their cost function.

The loss function is L=wE, where E=(G-Gest)^2 and G=F'F

The derivative of the loss function wrt F is proportional to F'(G-Gest)

Can't seem to figure it out.

Thanks

Emma

 

[fresh_42]

I have some trouble to understand you:

Do all functions depend on, say time ##t##, which the primes refer to? And why isn't ##G-G=0##? I first thought it could be the strange notation of a function, but then you defined a single ##G##. And last, could it be ##L \propto F(G-G)'##?

 

[emmasaunders12]

I have some trouble to understand you:

Do all functions depend on, say time ##t##, which the primes refer to? And why isn't ##G-G=0##? I first thought it could be the strange notation of a function, but then you defined a single ##G##. And last, could it be ##L \propto F(G-G)'##?

Thanks for the response, its the loss function of a neural network, so I've corrected to G and Gest, primes refer to transpose

 

[PeroK]

Thanks for the response, its the loss function of a neural network, so I've corrected to G and Gest, primes refer to transpose

Perhaps someone else can help, but without a lot more context I have no idea what mathematically we are dealing with here.

 

[emmasaunders12]

Perhaps someone else can help, but without a lot more context I have no idea what mathematically we are dealing with here.

The specific problem is described on page 4 here https://arxiv.org/pdf/1505.07376v3.pdf