Skip Navigation

The GPT-3 Architecture, on a Napkin

Inside the GPT-3

2

You're viewing a single thread.

2 comments
  • I've got a background in deep learning and I still struggle to understand the attention mechanism. I know it's a key/value store but I'm not sure what it's doing to the tensor when it passes through different layers.