I would use a damp wrung-out rag with just a tiny bit of soap in it too wipe it down. I might come back with clean water, still wrung-out.
Wooden toothpicks are great for the cracks because they are softer than the plastic.
I would save the IPA on a swab for any electric contacts, assuming you have removed batteries. If you can't remove the batteries, I would wait until you are having physical connection issues or have spilled something on it.
Step one is prevention. Wash my hands before playing. No eating with cheeto fingers, &c. To clean, I use diluted dish soap on a rag. Very wrung out so there's no chance of getting the internals wet. Wipe it on, let it sit for a couple seconds, wipe it off with a different damp cloth. Use clean, damp cloths until all the soapy shit is off (part of the reason I dilute it). If there's something sticky, like say that sticker on the back of the ps4 controller has fallen off, I've got adhesive remover (I have Rx strength stuff I use but orange oil is great, same method) before I go over it with the previous process. Rarely need to use the alcohol wipes but each controller gets that a few times a year.
Tldr, usually alcohol is the gold standard of cleaning electronics, but soap removes the oils that come offa my hands better. I just do it carefully. Ymmv.