Grouped Query Attention, Rotary Embedding, KV Cache, Root Mean Square Normalization
From GPT-3 to GPT-3.5-Turbo: Understanding the Latest Upgrades in OpenAI’s Language Model API.
No need to call in the Data Scientist to perform NLP