The top track can be assumed to be of infinite length, but for the bottom track this is not enough - to fit ℵ people on it, they'll have to be infinitely compressed. And since they are compressed - they are already dead. I'm not pulling the lever - preventing the (farther) desecration of corpses does not merit killing people who are still alive.
As a mathematician, this strikes me as an entirely reasonable interpretation, except for the fact that the compressed bodies would form a black hole, killing everyone regardless.
So you're correct to say that you wouldn't pull the lever, but your reasoning still missed an important detail.