Skip to main content

OpenAI

LiteLLM supports OpenAI Chat + Embedding calls.

Required API Keys

import os 
os.environ["OPENAI_API_KEY"] = "your-api-key"

Usage

import os 
from litellm import completion

os.environ["OPENAI_API_KEY"] = "your-api-key"

# openai call
response = completion(
model = "gpt-4o",
messages=[{ "content": "Hello, how are you?","role": "user"}]
)

Usage - LiteLLM Proxy Server

Here's how to call OpenAI models with the LiteLLM Proxy Server

1. Save key in your environment

export OPENAI_API_KEY=""

2. Start the proxy

model_list:
- model_name: gpt-3.5-turbo
litellm_params:
model: openai/gpt-3.5-turbo # The `openai/` prefix will call openai.chat.completions.create
api_key: os.environ/OPENAI_API_KEY
- model_name: gpt-3.5-turbo-instruct
litellm_params:
model: text-completion-openai/gpt-3.5-turbo-instruct # The `text-completion-openai/` prefix will call openai.completions.create
api_key: os.environ/OPENAI_API_KEY

3. Test it

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}
'

Optional Keys - OpenAI Organization, OpenAI API Base

import os 
os.environ["OPENAI_ORGANIZATION"] = "your-org-id" # OPTIONAL
os.environ["OPENAI_API_BASE"] = "openaiai-api-base" # OPTIONAL

OpenAI Chat Completion Models

Model NameFunction Call
o1-miniresponse = completion(model="o1-mini", messages=messages)
o1-previewresponse = completion(model="o1-preview", messages=messages)
gpt-4o-miniresponse = completion(model="gpt-4o-mini", messages=messages)
gpt-4o-mini-2024-07-18response = completion(model="gpt-4o-mini-2024-07-18", messages=messages)
gpt-4oresponse = completion(model="gpt-4o", messages=messages)
gpt-4o-2024-08-06response = completion(model="gpt-4o-2024-08-06", messages=messages)
gpt-4o-2024-05-13response = completion(model="gpt-4o-2024-05-13", messages=messages)
gpt-4-turboresponse = completion(model="gpt-4-turbo", messages=messages)
gpt-4-turbo-previewresponse = completion(model="gpt-4-0125-preview", messages=messages)
gpt-4-0125-previewresponse = completion(model="gpt-4-0125-preview", messages=messages)
gpt-4-1106-previewresponse = completion(model="gpt-4-1106-preview", messages=messages)
gpt-3.5-turbo-1106response = completion(model="gpt-3.5-turbo-1106", messages=messages)
gpt-3.5-turboresponse = completion(model="gpt-3.5-turbo", messages=messages)
gpt-3.5-turbo-0301response = completion(model="gpt-3.5-turbo-0301", messages=messages)
gpt-3.5-turbo-0613response = completion(model="gpt-3.5-turbo-0613", messages=messages)
gpt-3.5-turbo-16kresponse = completion(model="gpt-3.5-turbo-16k", messages=messages)
gpt-3.5-turbo-16k-0613response = completion(model="gpt-3.5-turbo-16k-0613", messages=messages)
gpt-4response = completion(model="gpt-4", messages=messages)
gpt-4-0314response = completion(model="gpt-4-0314", messages=messages)
gpt-4-0613response = completion(model="gpt-4-0613", messages=messages)
gpt-4-32kresponse = completion(model="gpt-4-32k", messages=messages)
gpt-4-32k-0314response = completion(model="gpt-4-32k-0314", messages=messages)
gpt-4-32k-0613response = completion(model="gpt-4-32k-0613", messages=messages)

These also support the OPENAI_API_BASE environment variable, which can be used to specify a custom API endpoint.

OpenAI Vision Models

Model NameFunction Call
gpt-4oresponse = completion(model="gpt-4o", messages=messages)
gpt-4-turboresponse = completion(model="gpt-4-turbo", messages=messages)
gpt-4-vision-previewresponse = completion(model="gpt-4-vision-preview", messages=messages)

Usage

import os 
from litellm import completion

os.environ["OPENAI_API_KEY"] = "your-api-key"

# openai call
response = completion(
model = "gpt-4-vision-preview",
messages=[
{
"role": "user",
"content": [
{
"type": "text",
"text": "What’s in this image?"
},
{
"type": "image_url",
"image_url": {
"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
}
}
]
}
],
)

OpenAI Fine Tuned Models

Model NameFunction Call
fine tuned gpt-4-0613response = completion(model="ft:gpt-4-0613", messages=messages)
fine tuned gpt-4o-2024-05-13response = completion(model="ft:gpt-4o-2024-05-13", messages=messages)
fine tuned gpt-3.5-turbo-0125response = completion(model="ft:gpt-3.5-turbo-0125", messages=messages)
fine tuned gpt-3.5-turbo-1106response = completion(model="ft:gpt-3.5-turbo-1106", messages=messages)
fine tuned gpt-3.5-turbo-0613response = completion(model="ft:gpt-3.5-turbo-0613", messages=messages)

Advanced

Getting OpenAI API Response Headers

Set litellm.return_response_headers = True to get raw response headers from OpenAI

You can expect to always get the _response_headers field from litellm.completion(), litellm.embedding() functions

litellm.return_response_headers = True

# /chat/completion
response = completion(
model="gpt-4o-mini",
messages=[
{
"role": "user",
"content": "hi",
}
],
)
print(f"response: {response}")
print("_response_headers=", response._response_headers)
Expected Response Headers from OpenAI
{
"date": "Sat, 20 Jul 2024 22:05:23 GMT",
"content-type": "application/json",
"transfer-encoding": "chunked",
"connection": "keep-alive",
"access-control-allow-origin": "*",
"openai-model": "text-embedding-ada-002",
"openai-organization": "*****",
"openai-processing-ms": "20",
"openai-version": "2020-10-01",
"strict-transport-security": "max-age=15552000; includeSubDomains; preload",
"x-ratelimit-limit-requests": "5000",
"x-ratelimit-limit-tokens": "5000000",
"x-ratelimit-remaining-requests": "4999",
"x-ratelimit-remaining-tokens": "4999999",
"x-ratelimit-reset-requests": "12ms",
"x-ratelimit-reset-tokens": "0s",
"x-request-id": "req_cc37487bfd336358231a17034bcfb4d9",
"cf-cache-status": "DYNAMIC",
"set-cookie": "__cf_bm=E_FJY8fdAIMBzBE2RZI2.OkMIO3lf8Hz.ydBQJ9m3q8-1721513123-1.0.1.1-6OK0zXvtd5s9Jgqfz66cU9gzQYpcuh_RLaUZ9dOgxR9Qeq4oJlu.04C09hOTCFn7Hg.k.2tiKLOX24szUE2shw; path=/; expires=Sat, 20-Jul-24 22:35:23 GMT; domain=.api.openai.com; HttpOnly; Secure; SameSite=None, *cfuvid=SDndIImxiO3U0aBcVtoy1TBQqYeQtVDo1L6*Nlpp7EU-1721513123215-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None",
"x-content-type-options": "nosniff",
"server": "cloudflare",
"cf-ray": "8a66409b4f8acee9-SJC",
"content-encoding": "br",
"alt-svc": "h3=\":443\"; ma=86400"
}

Parallel Function calling

See a detailed walthrough of parallel function calling with litellm here

import litellm
import json
# set openai api key
import os
os.environ['OPENAI_API_KEY'] = "" # litellm reads OPENAI_API_KEY from .env and sends the request
# Example dummy function hard coded to return the same weather
# In production, this could be your backend API or an external API
def get_current_weather(location, unit="fahrenheit"):
"""Get the current weather in a given location"""
if "tokyo" in location.lower():
return json.dumps({"location": "Tokyo", "temperature": "10", "unit": "celsius"})
elif "san francisco" in location.lower():
return json.dumps({"location": "San Francisco", "temperature": "72", "unit": "fahrenheit"})
elif "paris" in location.lower():
return json.dumps({"location": "Paris", "temperature": "22", "unit": "celsius"})
else:
return json.dumps({"location": location, "temperature": "unknown"})

messages = [{"role": "user", "content": "What's the weather like in San Francisco, Tokyo, and Paris?"}]
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
},
}
]

response = litellm.completion(
model="gpt-3.5-turbo-1106",
messages=messages,
tools=tools,
tool_choice="auto", # auto is default, but we'll be explicit
)
print("\nLLM Response1:\n", response)
response_message = response.choices[0].message
tool_calls = response.choices[0].message.tool_calls

Setting extra_headers for completion calls

import os 
from litellm import completion

os.environ["OPENAI_API_KEY"] = "your-api-key"

response = completion(
model = "gpt-3.5-turbo",
messages=[{ "content": "Hello, how are you?","role": "user"}],
extra_headers={"AI-Resource Group": "ishaan-resource"}
)

Setting Organization-ID for completion calls

This can be set in one of the following ways:

  • Environment Variable OPENAI_ORGANIZATION
  • Params to litellm.completion(model=model, organization="your-organization-id")
  • Set as litellm.organization="your-organization-id"
import os 
from litellm import completion

os.environ["OPENAI_API_KEY"] = "your-api-key"
os.environ["OPENAI_ORGANIZATION"] = "your-org-id" # OPTIONAL

response = completion(
model = "gpt-3.5-turbo",
messages=[{ "content": "Hello, how are you?","role": "user"}]
)

Set ssl_verify=False

This is done by setting your own httpx.Client

  • For litellm.completion set litellm.client_session=httpx.Client(verify=False)
  • For litellm.acompletion set litellm.aclient_session=AsyncClient.Client(verify=False)
import litellm, httpx

# for completion
litellm.client_session = httpx.Client(verify=False)
response = litellm.completion(
model="gpt-3.5-turbo",
messages=messages,
)

# for acompletion
litellm.aclient_session = httpx.AsyncClient(verify=False)
response = litellm.acompletion(
model="gpt-3.5-turbo",
messages=messages,
)

Using Helicone Proxy with LiteLLM

import os 
import litellm
from litellm import completion

os.environ["OPENAI_API_KEY"] = ""

# os.environ["OPENAI_API_BASE"] = ""
litellm.api_base = "https://oai.hconeai.com/v1"
litellm.headers = {
"Helicone-Auth": f"Bearer {os.getenv('HELICONE_API_KEY')}",
"Helicone-Cache-Enabled": "true",
}

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion("gpt-3.5-turbo", messages)

Using OpenAI Proxy with LiteLLM

import os 
import litellm
from litellm import completion

os.environ["OPENAI_API_KEY"] = ""

# set custom api base to your proxy
# either set .env or litellm.api_base
# os.environ["OPENAI_API_BASE"] = ""
litellm.api_base = "your-openai-proxy-url"


messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion("openai/your-model-name", messages)

If you need to set api_base dynamically, just pass it in completions instead - completions(...,api_base="your-proxy-api-base")

For more check out setting API Base/Keys

Forwarding Org ID for Proxy requests

Forward openai Org ID's from the client to OpenAI with forward_openai_org_id param.

  1. Setup config.yaml
model_list:
- model_name: "gpt-3.5-turbo"
litellm_params:
model: gpt-3.5-turbo
api_key: os.environ/OPENAI_API_KEY

general_settings:
forward_openai_org_id: true # 👈 KEY CHANGE
  1. Start Proxy
litellm --config config.yaml --detailed_debug

# RUNNING on http://0.0.0.0:4000
  1. Make OpenAI call
from openai import OpenAI
client = OpenAI(
api_key="sk-1234",
organization="my-special-org",
base_url="http://0.0.0.0:4000"
)

client.chat.completions.create(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello world"}])

In your logs you should see the forwarded org id

LiteLLM:DEBUG: utils.py:255 - Request to litellm:
LiteLLM:DEBUG: utils.py:255 - litellm.acompletion(... organization='my-special-org',)