The answer I got on reddit, that helped me understand it.
The total energy is given by
(1/2μ_0)∫B2dV
where the integral is taken over all space. Let us now split the space into a region X which contains the solenoid, and a region Y which is everything else. We can now write the integral as...