Problem: Find a (limited?) solution to the diff eq.

At the end of the solution, when you transform \(\displaystyle \frac{-1}{s+1} + \frac{2}{s-3}\)
why doesnt it become \(\displaystyle -e^{-t} + 2e^{3t} \), t>0 ?

That would be correct if the right hand side of the equation were 0. But the delta functions on the right mean that those are correct only for x greater than certain values so the step functions are needed.