Zero-Shot, Few Shot, and Chain-of-thought Prompt

What is Prompting ?

In natural language processing, prompting is telling a language model what to do or how to respond in response to examples or instructions. In this domain, different methodologies like few-shot, zero-shot, and chain-of-thought prompting are critical to improving the performance of AI models such as GPT-3.5 Turbo. From adaptability and contextual understanding to logical reasoning and task-specific learning, each technique has its own advantages. Within this framework, we explore the nuances of each prompting technique and provide examples and coding demonstrations to highlight how to use them. Investigating these methods helps us understand the adaptability and potential of AI models and demonstrates how well they work to solve a variety of problems in various fields.

Zero Shot Prompting

Zero-shot prompting involves instructing a language model without providing specific examples, relying on the model’s inherent understanding of the context. The language model, having pre-existing knowledge, generates a response based on its contextual understanding.

Prompt:

Perform sentiment analysis on the given text and categorize it as neutral, negative, or positive.

Text Input:

The Recent trip was great

Output:

Sentiment: Positive

In this case, the language model is tasked with sentiment analysis without being given specific instances for training. The model leverages its existing knowledge to interpret and categorize the sentiment of the provided text. This approach is known as zero-shot prompting, as it relies on the model’s innate contextual comprehension rather than explicit examples.

Chain-of-Thought Prompting

Before delving directly into few-shot prompting, let me introduce you to a chain of thoughts. This will help you understand why providing examples within the prompt is necessary, especially in complex scenarios.

Standard Prompt:

Question 1: Sam has 3 notebooks. He purchases 4 more packs of notebooks, with 5 notebooks in each pack. How many notebooks does Sam have now?

Answer: 23 notebooks

Question 2: The basket has 30 apples. If 20 apples are used for lunch and an additional 6 are bought from the market, how many apples are there in the basket?

The Model Output will be 11 which is wrong

Chain-of-Thought Prompt:

Question 1: Sam has 3 notebooks. He purchases 4 more packs of notebooks, with 5 notebooks in each pack. How many notebooks does Sam have now?

Answer: Sam initial notebooks: 3 notebooks
Notebooks from the new packs: 4 packs * 5 notebooks/pack = 20 notebooks

Now, add these two quantities:

3 notebooks (initial) + 20 notebooks (new packs) = 23 notebooks

Question 2: The basket has 30 apples. If 20 apples are used for lunch and an additional 6 are bought from the market, how many apples are there in the basket?

The Model Output will be:

Initial number of apples in the basket: 30 apples
Apples used for lunch: -20 apples
Apples brought from the market: +6 apples

Now, add these quantities:

30 apples (initial) — 20 apples (used for lunch) + 6 apples (brought from the market) = 16 apples

Chain-of-thought (CoT) prompting means guiding a language model through a series of connected logical steps or thoughts. It allows the model to think step by step, considering different aspects of a problem.

When we say complex reasoning capabilities, we mean the ability of the model to think through intricate or complicated scenarios.

Now, few-shot prompting involves providing a few examples or instances within the prompt to help the model understand the task better.

So, combining chain-of-thought with few-shot prompting means you guide the model through logical steps, allowing it to reason through complexities, and you also provide a few examples to enhance its understanding. This combination helps the model tackle more intricate tasks that require thoughtful reasoning before generating a response.

One-Shot and Few Shot Prompting:

Even though large-language models can understand and generate text without examples (zero-shot capabilities), they struggle with more challenging tasks in this setting. To address this, we use a technique called few-shot prompting. This involves providing the model with a few examples or demonstrations in the prompt, essentially giving it a bit of training to improve its performance. These examples act as training guides, helping the model learn and generate better responses in similar situations.

Prompt:

A blizzlenut is a rare fruit found in the mystical forest.

An example of a sentence that uses the word blizzlenut is: During our magical journey, we discovered a grove filled with enchanting blizzlenuts.

To perform a gleamfrolic means to twirl around with joy.

Output:

An example of a sentence that uses the word gleamfrolic is: After receiving good news, she couldn’t help but gleamfrolic in the garden.

We notice that the model can grasp a task with just a single example (one-shot). However, when dealing with more challenging tasks, we can explore improved performance by experimenting with an increased number of examples, such as 3-shot, 5-shot, 10-shot, and so on.

Let’s try for Few-Shot Prompt for Text Sentiments:

This is fantastic! — — ->Negative

This is terrible! — — ->Positive

Wow, that book was amazing! — — ->Negative

What a dreadful performance! — — ->Positive

I have randomly assigned sentiments to the above texts.

Output:

This dessert is delicious! — — ->Negative

Let’s Dive into the Coding Part

Zero Shot Prompt Example:

from openai import OpenAI
import re
import os
import pandas as pd
client = OpenAI()

def sentiment_analysis_example(text):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[
            {
                "role": "system",
                "content": 'Perform sentiment analysis on the given text: "{text}". Classify it as either Positive, Negative, or Neutral sentiment. Provide the result in JSON format with the following keys: sentiment.'
            },
            {
                "role": "user",
                "content": f'Analyze the sentiment of the text "{text}" and categorize it as Positive, Negative, or Neutral. Return the result in JSON format with the key "sentiment".'
            }
        ],
        max_tokens=150,
        n=1,
        stop=None,
        temperature=0
    )

    result = response.choices[0].message.content
    try:
        start_index = result.find('{')
        end_index = result.rfind('}')

        if start_index != -1 and end_index != -1:
            result = result[start_index:end_index + 1]
    except:
        pass
    print(result)

example_text = "This is fantastic!"
sentiment_analysis_example(example_text)

Output:

{
  "sentiment": "Positive"
}

In this example, we used a method called zero-shot prompting to ask the model to understand the feeling behind text, even though it wasn’t trained specifically for that task. This shows how flexible the model is across different situations. We told the GPT-3.5 Turbo model to figure out whether a given text was positive, negative, or neutral without teaching it how to do that beforehand. Surprisingly, it correctly said the sentiment was Positive for the phrase “This is fantastic!” This proves that the model can understand emotions in text without needing previous training on sentiment analysis.

Chain-of-Thought Prompt Example:

from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
    {
      "role": "system",
      "content": "Determine the cost"
    },
    {
      "role": "user",
      "content": "Sam has 3 notebooks. He purchases 4 more packs of notebooks, with 5 notebooks in each pack. How many notebooks does Sam have now?"
    },
    {
      "role": "assistant",
      "content": "Sam initial notebooks: 3 notebooks\nNotebooks from the new packs: 4 packs * 5 notebooks/pack = 20 notebooks\nNow, add these two quantities:\n3 notebooks (initial) + 20 notebooks (new packs) = 23 notebooks"
    },
    {
      "role": "user",
      "content": "The basket has 30 apples. If 20 apples are used for lunch and an additional 6 are bought from the market, how many apples are there in the basket?"
    }
  ],
  temperature=0,
  max_tokens=1024,
  top_p=1,
  frequency_penalty=0,
  presence_penalty=0
)
result = response.choices[0].message.content
print(result)

Output:

Apples in the basket: 30 apples
Apples used for lunch: 20 apples
Apples bought from the market: 6 apples
Now, subtract the apples used and add the apples bought:
30 apples - 20 apples + 6 apples = 16 apples

The previous prompt involved a complex problem-solving scenario where Sam’s notebook count and Sarah’s apples in the basket were determined through a series of calculations. The assistant broke down the problems, provided step-by-step explanations, and computed the final results.

In subsequent prompts, the model can reference this previous solution to guide its responses. For example, it can leverage the understanding gained from solving the notebook and apple problems to tackle similar questions efficiently. This means that if faced with a new prompt related to counting items or performing arithmetic operations, the model can draw upon its learned knowledge to generate accurate and detailed responses, showcasing its ability to apply acquired reasoning skills to solve related problems.

Note: As the complexity or size of the prompt increases, so does its associated cost.

Conclusion:

• Each prompting technique serves distinct purposes in harnessing the capabilities of AI models like GPT-3.5 Turbo.
• Zero-shot prompting demonstrates the model’s adaptability and contextual understanding.
• Chain-of-thought prompting promotes transparency and logical reasoning, aiding in complex problem-solving.
• Few-shot prompting facilitates task-specific learning and improved performance through example-based training.
• Collectively, these prompting techniques showcase the versatility and potential of AI models, paving the way for innovative applications across various domains.