ChatGPT is powered by gpt-3.5-turbo and gpt-4, OpenAI's most advanced models.
You can build your own applications with gpt-3.5-turbo or gpt-4 using the OpenAI API.
Chat models take a series of messages as input, and return an AI-written message as output.
This guide illustrates the chat format with a few example API calls.
# if needed, install and/or upgrade to the latest version of the OpenAI Python library
%pip install --upgrade openai
# import the OpenAI Python library for calling the OpenAI API
import openai
A chat API call has two required inputs:
model: the name of the model you want to use (e.g., gpt-3.5-turbo, gpt-4, gpt-3.5-turbo-0613, gpt-3.5-turbo-16k-0613)messages: a list of message objects, where each object has two required fields:
role: the role of the messenger (either system, user, or assistant)content: the content of the message (e.g., Write me a beautiful poem)Messages can also contain an optional name field, which give the messenger a name. E.g., example-user, Alice, BlackbeardBot. Names may not contain spaces.
As of June 2023, you can also optionally submit a list of functions that tell GPT whether it can generate JSON to feed into a function. For details, see the documentation, API reference, or the Cookbook guide How to call functions with chat models.
Typically, a conversation will start with a system message that tells the assistant how to behave, followed by alternating user and assistant messages, but you are not required to follow this format.
Let's look at an example chat API calls to see how the chat format works in practice.
# Example OpenAI Python library request
MODEL = "gpt-3.5-turbo"
response = openai.ChatCompletion.create(
model=MODEL,
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Knock knock."},
{"role": "assistant", "content": "Who's there?"},
{"role": "user", "content": "Orange."},
],
temperature=0,
)
response
<OpenAIObject chat.completion id=chatcmpl-7UkgnSDzlevZxiy0YjZcLYdUMz5yZ at 0x118e394f0> JSON: {
"id": "chatcmpl-7UkgnSDzlevZxiy0YjZcLYdUMz5yZ",
"object": "chat.completion",
"created": 1687563669,
"model": "gpt-3.5-turbo-0301",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Orange who?"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 39,
"completion_tokens": 3,
"total_tokens": 42
}
}As you can see, the response object has a few fields:
id: the ID of the requestobject: the type of object returned (e.g., chat.completion)created: the timestamp of the requestmodel: the full name of the model used to generate the responseusage: the number of tokens used to generate the replies, counting prompt, completion, and totalchoices: a list of completion objects (only one, unless you set n greater than 1)
message: the message object generated by the model, with role and contentfinish_reason: the reason the model stopped generating text (either stop, or length if max_tokens limit was reached)index: the index of the completion in the list of choicesExtract just the reply with:
response['choices'][0]['message']['content']
'Orange who?'
Even non-conversation-based tasks can fit into the chat format, by placing the instruction in the first user message.
For example, to ask the model to explain asynchronous programming in the style of the pirate Blackbeard, we can structure conversation as follows:
# example with a system message
response = openai.ChatCompletion.create(
model=MODEL,
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain asynchronous programming in the style of the pirate Blackbeard."},
],
temperature=0,
)
print(response['choices'][0]['message']['content'])
Ahoy matey! Asynchronous programming be like havin' a crew o' pirates workin' on different tasks at the same time. Ye see, instead o' waitin' for one task to be completed before startin' the next, ye can assign tasks to yer crew and let 'em work on 'em simultaneously. This way, ye can get more done in less time and keep yer ship sailin' smoothly. It be like havin' a bunch o' pirates rowin' the ship at different speeds, but still gettin' us to our destination. Arrr!
# example without a system message
response = openai.ChatCompletion.create(
model=MODEL,
messages=[
{"role": "user", "content": "Explain asynchronous programming in the style of the pirate Blackbeard."},
],
temperature=0,
)
print(response['choices'][0]['message']['content'])
Ahoy mateys! Let me tell ye about asynchronous programming, arrr! It be like havin' a crew of sailors workin' on different tasks at the same time, without waitin' for each other to finish. Ye see, in traditional programming, ye have to wait for one task to be completed before movin' on to the next. But with asynchronous programming, ye can start multiple tasks at once and let them run in the background while ye focus on other things. It be like havin' a lookout keepin' watch for enemy ships while the rest of the crew be busy with their own tasks. They don't have to stop what they're doin' to keep an eye out, because the lookout be doin' it for them. And when the lookout spots an enemy ship, they can alert the crew and everyone can work together to defend the ship. In the same way, asynchronous programming allows different parts of yer code to work together without gettin' in each other's way. It be especially useful for tasks that take a long time to complete, like loadin' large files or connectin' to a server. Instead of makin' yer program wait for these tasks to finish, ye can let them run in the background while yer program continues to do other things. So there ye have it, me hearties! Asynchronous programming be like havin' a crew of sailors workin' together without gettin' in each other's way. It be a powerful tool for any programmer, and one that can help ye sail the seas of code with ease!
Best practices for instructing models may change from model version to model version. The advice that follows applies to gpt-3.5-turbo-0301 and may not apply to future models.
The system message can be used to prime the assistant with different personalities or behaviors.
Be aware that gpt-3.5-turbo-0301 does not generally pay as much attention to the system message as gpt-4-0314 or gpt-3.5-turbo-0613. Therefore, for gpt-3.5-turbo-0301, we recommend placing important instructions in the user message instead. Some developers have found success in continually moving the system message near the end of the conversation to keep the model's attention from drifting away as conversations get longer.
# An example of a system message that primes the assistant to explain concepts in great depth
response = openai.ChatCompletion.create(
model=MODEL,
messages=[
{"role": "system", "content": "You are a friendly and helpful teaching assistant. You explain concepts in great depth using simple terms, and you give examples to help people learn. At the end of each explanation, you ask a question to check for understanding"},
{"role": "user", "content": "Can you explain how fractions work?"},
],
temperature=0,
)
print(response["choices"][0]["message"]["content"])
Sure! Fractions are a way of representing a part of a whole. The top number of a fraction is called the numerator, and it represents how many parts of the whole we are talking about. The bottom number is called the denominator, and it represents how many equal parts the whole is divided into. For example, if we have a pizza that is divided into 8 equal slices, and we take 3 slices, we can represent that as the fraction 3/8. The numerator is 3 because we took 3 slices, and the denominator is 8 because the pizza was divided into 8 slices. To add or subtract fractions, we need to have a common denominator. This means that the denominators of the fractions need to be the same. To do this, we can find the least common multiple (LCM) of the denominators and then convert each fraction to an equivalent fraction with the LCM as the denominator. To multiply fractions, we simply multiply the numerators together and the denominators together. To divide fractions, we multiply the first fraction by the reciprocal of the second fraction (flip the second fraction upside down). Now, here's a question to check for understanding: If we have a pizza that is divided into 12 equal slices, and we take 4 slices, what is the fraction that represents how much of the pizza we took?
# An example of a system message that primes the assistant to give brief, to-the-point answers
response = openai.ChatCompletion.create(
model=MODEL,
messages=[
{"role": "system", "content": "You are a laconic assistant. You reply with brief, to-the-point answers with no elaboration."},
{"role": "user", "content": "Can you explain how fractions work?"},
],
temperature=0,
)
print(response["choices"][0]["message"]["content"])
Fractions represent a part of a whole. They consist of a numerator (top number) and a denominator (bottom number) separated by a line. The numerator represents how many parts of the whole are being considered, while the denominator represents the total number of equal parts that make up the whole.
In some cases, it's easier to show the model what you want rather than tell the model what you want.
One way to show the model what you want is with faked example messages.
For example:
# An example of a faked few-shot conversation to prime the model into translating business jargon to simpler speech
response = openai.ChatCompletion.create(
model=MODEL,
messages=[
{"role": "system", "content": "You are a helpful, pattern-following assistant."},
{"role": "user", "content": "Help me translate the following corporate jargon into plain English."},
{"role": "assistant", "content": "Sure, I'd be happy to!"},
{"role": "user", "content": "New synergies will help drive top-line growth."},
{"role": "assistant", "content": "Things working well together will increase revenue."},
{"role": "user", "content": "Let's circle back when we have more bandwidth to touch base on opportunities for increased leverage."},
{"role": "assistant", "content": "Let's talk later when we're less busy about how to do better."},
{"role": "user", "content": "This late pivot means we don't have time to boil the ocean for the client deliverable."},
],
temperature=0,
)
print(response["choices"][0]["message"]["content"])
We don't have enough time to complete the entire project perfectly.
To help clarify that the example messages are not part of a real conversation, and shouldn't be referred back to by the model, you can try setting the name field of system messages to example_user and example_assistant.
Transforming the few-shot example above, we could write:
# The business jargon translation example, but with example names for the example messages
response = openai.ChatCompletion.create(
model=MODEL,
messages=[
{"role": "system", "content": "You are a helpful, pattern-following assistant that translates corporate jargon into plain English."},
{"role": "system", "name":"example_user", "content": "New synergies will help drive top-line growth."},
{"role": "system", "name": "example_assistant", "content": "Things working well together will increase revenue."},
{"role": "system", "name":"example_user", "content": "Let's circle back when we have more bandwidth to touch base on opportunities for increased leverage."},
{"role": "system", "name": "example_assistant", "content": "Let's talk later when we're less busy about how to do better."},
{"role": "user", "content": "This late pivot means we don't have time to boil the ocean for the client deliverable."},
],
temperature=0,
)
print(response["choices"][0]["message"]["content"])
This sudden change in plans means we don't have enough time to do everything for the client's project.
Not every attempt at engineering conversations will succeed at first.
If your first attempts fail, don't be afraid to experiment with different ways of priming or conditioning the model.
As an example, one developer discovered an increase in accuracy when they inserted a user message that said "Great job so far, these have been perfect" to help condition the model into providing higher quality responses.
For more ideas on how to lift the reliability of the models, consider reading our guide on techniques to increase reliability. It was written for non-chat models, but many of its principles still apply.
When you submit your request, the API transforms the messages into a sequence of tokens.
The number of tokens used affects:
gpt-3.5-turbo or 8,192 for gpt-4)You can use the following function to count the number of tokens that a list of messages will use.
Note that the exact way that tokens are counted from messages may change from model to model. Consider the counts from the function below an estimate, not a timeless guarantee.
In particular, requests that use the optional functions input will consume extra tokens on top of the estimates calculated below.
Read more about counting tokens in How to count tokens with tiktoken.
import tiktoken
def num_tokens_from_messages(messages, model="gpt-3.5-turbo-0613"):
"""Return the number of tokens used by a list of messages."""
try:
encoding = tiktoken.encoding_for_model(model)
except KeyError:
print("Warning: model not found. Using cl100k_base encoding.")
encoding = tiktoken.get_encoding("cl100k_base")
if model in {
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-16k-0613",
"gpt-4-0314",
"gpt-4-32k-0314",
"gpt-4-0613",
"gpt-4-32k-0613",
}:
tokens_per_message = 3
tokens_per_name = 1
elif model == "gpt-3.5-turbo-0301":
tokens_per_message = 4 # every message follows <|start|>{role/name}\n{content}<|end|>\n
tokens_per_name = -1 # if there's a name, the role is omitted
elif "gpt-3.5-turbo" in model:
print("Warning: gpt-3.5-turbo may update over time. Returning num tokens assuming gpt-3.5-turbo-0613.")
return num_tokens_from_messages(messages, model="gpt-3.5-turbo-0613")
elif "gpt-4" in model:
print("Warning: gpt-4 may update over time. Returning num tokens assuming gpt-4-0613.")
return num_tokens_from_messages(messages, model="gpt-4-0613")
else:
raise NotImplementedError(
f"""num_tokens_from_messages() is not implemented for model {model}. See https://github.com/openai/openai-python/blob/main/chatml.md for information on how messages are converted to tokens."""
)
num_tokens = 0
for message in messages:
num_tokens += tokens_per_message
for key, value in message.items():
num_tokens += len(encoding.encode(value))
if key == "name":
num_tokens += tokens_per_name
num_tokens += 3 # every reply is primed with <|start|>assistant<|message|>
return num_tokens
# let's verify the function above matches the OpenAI API response
import openai
example_messages = [
{
"role": "system",
"content": "You are a helpful, pattern-following assistant that translates corporate jargon into plain English.",
},
{
"role": "system",
"name": "example_user",
"content": "New synergies will help drive top-line growth.",
},
{
"role": "system",
"name": "example_assistant",
"content": "Things working well together will increase revenue.",
},
{
"role": "system",
"name": "example_user",
"content": "Let's circle back when we have more bandwidth to touch base on opportunities for increased leverage.",
},
{
"role": "system",
"name": "example_assistant",
"content": "Let's talk later when we're less busy about how to do better.",
},
{
"role": "user",
"content": "This late pivot means we don't have time to boil the ocean for the client deliverable.",
},
]
for model in [
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4",
]:
print(model)
# example token count from the function defined above
print(f"{num_tokens_from_messages(example_messages, model)} prompt tokens counted by num_tokens_from_messages().")
# example token count from the OpenAI API
response = openai.ChatCompletion.create(
model=model,
messages=example_messages,
temperature=0,
max_tokens=1, # we're only counting input tokens here, so let's not waste tokens on the output
)
print(f'{response["usage"]["prompt_tokens"]} prompt tokens counted by the OpenAI API.')
print()
gpt-3.5-turbo-0301 127 prompt tokens counted by num_tokens_from_messages(). 127 prompt tokens counted by the OpenAI API. gpt-3.5-turbo-0613 129 prompt tokens counted by num_tokens_from_messages(). 129 prompt tokens counted by the OpenAI API. gpt-3.5-turbo Warning: gpt-3.5-turbo may update over time. Returning num tokens assuming gpt-3.5-turbo-0613. 129 prompt tokens counted by num_tokens_from_messages(). 127 prompt tokens counted by the OpenAI API. gpt-4-0314 129 prompt tokens counted by num_tokens_from_messages(). 129 prompt tokens counted by the OpenAI API. gpt-4-0613 129 prompt tokens counted by num_tokens_from_messages(). 129 prompt tokens counted by the OpenAI API. gpt-4 Warning: gpt-4 may update over time. Returning num tokens assuming gpt-4-0613. 129 prompt tokens counted by num_tokens_from_messages(). 129 prompt tokens counted by the OpenAI API.