Keep in mind that there will often be no fruit. Pawpaws spread in monoclonal colonies, and a colony can't pollinate itself. I live surrounded by many pawpaw groves, but only a few actually make fruit. Supposedly you can spray the trees down with fish emulsion to attract the flies that pollinate them, but that still only helps if you have 2 non-clones next to each other
Finding them is pretty easy once you know what to look for, but until you've seen them, it's hard. Easiest time to find them as a novice is in the spring. They have very unique flowers that pop out before there are any leaves. Also, in the late summer/ fall, easiest way to find them is by smell. The fruit is so fragrant, if you know the smell, you can find the tree.
They have those purple flowers that hang downward, right?
I've seen some pawpaw saplings in pots before, but maybe I should have gotten a closer look, haha! I guess that doesn't necessarily translate to how they'll look in the wild. Hopefully I'll have a chance to get the smell down at some point, haha!
Here's a few more things to look out for. They are fine growing in shade. They usually are in groups (clonal colonies, technically) unlike some similar looking trees. Deer don't eat them, so they do really well in areas with an overabundance of deer that eat all their competition.