Welcome to our community

Be a part of something great, join today!

Why isn't the "l1"-norm differentiable?

OhMyMarkov

Member
Mar 5, 2012
83
Hello everyone!

I've searched a lot for this one, but couldn't find an answer:

If x is in R^N then ||x||_0 = x_1 + x_2 + ... + x_N. Why, then, isn't this norm differentiable?

(Btw, how to make LaTeX work?)

Thank you!
 

Jester

Well-known member
MHB Math Helper
Jan 26, 2012
183
Hello everyone!

I've searched a lot for this one, but couldn't find an answer:

If x is in R^N then ||x||_0 = x_1 + x_2 + ... + x_N. Why, then, isn't this norm differentiable?

(Btw, how to make LaTeX work?)

Thank you!
Put your math in between $\$$ signs. For example

$\$$||x||_0 = x_1 + x_2 + ... + x_N$\$ $ gives $ ||x||_0 = x_1 + x_2 + ... + x_N $.
 

Opalg

MHB Oldtimer
Staff member
Feb 7, 2012
2,680
Hello everyone!

I've searched a lot for this one, but couldn't find an answer:

If x is in R^N then ||x||_0 = x_1 + x_2 + ... + x_N. Why, then, isn't this norm differentiable?
I think you mean $\|x\|_0 = |x_1| + |x_2| + ... + |x_N|$ (without the absolute values it is not a norm). The geometric reason for non-differentiability of the norm is that the unit sphere $\{x:\|x\|_0=1\}$ has "corners". In one dimension, the function $f(x)=|x|$ is not differentiable at $x=0$. In $R^N$, the function $\|x\|_0$ is not differentiable at an extreme point of the unit sphere, such as the point $(1,0,\ldots,0).$
 

OhMyMarkov

Member
Mar 5, 2012
83
Ohh, I missed the part with the absolute values... It's all clear now
 

Venki

New member
Jul 2, 2012
1
I think you mean $\|x\|_0 = |x_1| + |x_2| + ... + |x_N|$ (without the absolute values it is not a norm). The geometric reason for non-differentiability of the norm is that the unit sphere $\{x:\|x\|_0=1\}$ has "corners". In one dimension, the function $f(x)=|x|$ is not differentiable at $x=0$. In $R^N$, the function $\|x\|_0$ is not differentiable at an extreme point of the unit sphere, such as the point $(1,0,\ldots,0).$

Hi Opalg,

I like your explanation from geometric point of view. It is very helpful. But Still it is confusing in terms of matrix calculus. could you elaborate the answer? Please.

Thank you
Venki
 
Last edited:

CaptainBlack

Well-known member
Jan 26, 2012
890
Hi Opalg,

I like your explanation from geometric point of view. It is very helpful. But Still it is confusing in terms of matrix calculus. could you elaborate the answer? Please.

Thank you
Venki
It is because |x| is not differentiable at the corner x=0.

CB