Understanding Llama2: KV Cache, Grouped Query Attention, Rotary Embedding and More

Grouped Query Attention, Rotary Embedding, KV Cache, Root Mean Square Normalization

Fine-Tuning Vs RAG in Generative AI

Vanishing Gradient Problem in RNNs

Beginner’s Guide to OpenAI’s GPT-3.5-Turbo Model

From GPT-3 to GPT-3.5-Turbo: Understanding the Latest Upgrades in OpenAI’s Language Model API.

Creating Your Own ChatGPT: A Guide to Fine-Tuning LLMs with LoRA

As AI continues to evolve, chatbots are becoming increasingly sophisticated and human-like in their ability to respond to natural language inputs. One of the most impressive models in this space is…

Leave One Subject Out Cross Validation for Machine Learning Models

How to implement cross validation in Python from scratch.

What is Gaussian Noise in Deep Learning? How and Why it is used?

Visualizing Attention in Vision Transformers

Knowledge Distillation, aka Teacher-Student Model

With the release of large models in the last few years, from GPT-3 to Megatron, I keep pondering how to experiment and use these models for a specific use case. These models are trained on massive…

Understand collate_fn in PyTorch

A beginner-friendly tutorial on how to create batches in PyTorch and how to modify default behavior for our needs.