Posted 3/15/2024, 8:56:31 PM
Developer Squeezes GPT-2 Into Excel to Demonstrate Language Model
- Developer crams GPT-2 into a working Excel spreadsheet to educate about how language models work
- Uses 10 tokens for input to show predictions, much less than GPT-4 Turbo's 128,000
- Hardest part was tokenization because it involves text processing not just math
- OpenAI released GPT-2 in 2019 but initially feared it could generate abusive text
- Runs "GPT-2 Small" with 124 million parameters, compared to 175 billion in GPT-3