How to fix my pants so I don't look like I have a boner when I sit down?

Most of my pants have gotten big on me, but it feels wrong to go out and buy new ones when these work fine, so I've been wearing them with a belt anyway. The problem is that when I sit down they bunch up around my crotch and it looks like I have a massive boner when I don't. Obviously, I don't want this to keep happening and am wondering if anybody knows of a way to fix this. I have relatively little expierence with sewing or tailoring but I do have access to a sewing machine. Any advice would be appreciated.