What direction do flags blow on a tall ship?

I just finished building a model of a pirate ship and want the flags to be pointing the right direction. Should the flags be blowing to the bow because the wind is behind the ship, or to the stern as the ship drags the sails along?

Looking at paintings they seem to mostly be blowing to the stern, however there are some that have them blowing to the bow aswell; so I figured it'd be best to ask people who have experience.