I think it'll give you some pictures that it knows the answers to, just to actually make sure you're not a robot, but if you get that picture correct, it'll serve you a picture that it doesn't know what the correct answers are, and you teach it by answering what it assumes it's correctly.
I wonder how they pick these pictures. They must have some sort of program that picks pictures from street view including the thing they're looking for. I can't imagine they're doing it all manually
Step 1: collect pictures that may contain objects you want to identify
Step 2: add some manually identified control pictures
Step 3: ask users to identify the object in a sample of the potential pictures using the control pictures for verification
So i used to dabble in doing jobs for Amazon’s Mturk work system to try and make some extra money. I’m not gonna go into the details of what that is or how it works here, but we would get a lot of little jobs that could be “write out closed captions for these short videos”, OR “select any squares of this image that meet the request”, like on this post. We’d be allowed to not select any squares if that was the case. They always looked like screen shots from google earth or something. A lot of times, like here, the solution was not in the provided image.
I always felt like us Mturk human workers were providing the solutions to these “anti bot” measures.