Size is a poor indicator. 11 of me could take down a large but slow grazing animal. But a smaller predator would jack up 11 unarmed me's.
Maybe we could all gnaw on a cow long enough to bring it down.
With no weapons, I find this unlikely. The buffalo would pulverize you one by one and there's not much the rest of you could do to harm it. If you crowd together it could trample you all in one go.
It's an "open field", so if there's a cliff it's probably not very nearby. A buffalo is a lot faster than a human so I'd expect it to finish all of him of before they could get far.