Why isn't the "l1"-norm differentiable?

[OhMyMarkov]

Hello everyone!

I've searched a lot for this one, but couldn't find an answer:

If x is in R^N then ||x||_0 = x_1 + x_2 + ... + x_N. Why, then, isn't this norm differentiable?

(Btw, how to make LaTeX work?)

Thank you!

 

[Jester]

Hello everyone!

I've searched a lot for this one, but couldn't find an answer:

If x is in R^N then ||x||_0 = x_1 + x_2 + ... + x_N. Why, then, isn't this norm differentiable?

(Btw, how to make LaTeX work?)

Thank you!

Put your math in between $\$$ signs. For example

$\$$||x||_0 = x_1 + x_2 + ... + x_N$\$ $ gives $ ||x||_0 = x_1 + x_2 + ... + x_N $.

 

[Opalg]

Hello everyone!

I've searched a lot for this one, but couldn't find an answer:

If x is in R^N then ||x||_0 = x_1 + x_2 + ... + x_N. Why, then, isn't this norm differentiable?

I think you mean $\|x\|_0 = |x_1| + |x_2| + ... + |x_N|$ (without the absolute values it is not a norm). The geometric reason for non-differentiability of the norm is that the unit sphere $\{x:\|x\|_0=1\}$ has "corners". In one dimension, the function $f(x)=|x|$ is not differentiable at $x=0$. In $R^N$, the function $\|x\|_0$ is not differentiable at an extreme point of the unit sphere, such as the point $(1,0,\ldots,0).$

 

[OhMyMarkov]

Ohh, I missed the part with the absolute values... It's all clear now

 

[Venki]

I think you mean $\|x\|_0 = |x_1| + |x_2| + ... + |x_N|$ (without the absolute values it is not a norm). The geometric reason for non-differentiability of the norm is that the unit sphere $\{x:\|x\|_0=1\}$ has "corners". In one dimension, the function $f(x)=|x|$ is not differentiable at $x=0$. In $R^N$, the function $\|x\|_0$ is not differentiable at an extreme point of the unit sphere, such as the point $(1,0,\ldots,0).$

Hi Opalg,

I like your explanation from geometric point of view. It is very helpful. But Still it is confusing in terms of matrix calculus. could you elaborate the answer? Please.

Thank you
Venki

 

[CaptainBlack]

Hi Opalg,

I like your explanation from geometric point of view. It is very helpful. But Still it is confusing in terms of matrix calculus. could you elaborate the answer? Please.

Thank you
Venki

It is because |x| is not differentiable at the corner x=0.

CB