Ollama API in Python, and a Hosted Alternative

The short version

If you search for ollama api python, you usually want one of three things: call a model running on your own machine, stream tokens from it, or make the same script work somewhere else. Ollama gives you all three. It runs a local server, by default at http://localhost:11434, and you talk to it with the ollama Python package, with plain HTTP, or with any OpenAI client pointed at its /v1 path.

The catch is that the model has to run on your hardware. A laptop without a decent GPU will be slow with larger models, and a server without one is a poor fit. For that case a hosted OpenAI-compatible API is the easier route, and because the request format is the same, switching costs you three lines of configuration. Both paths are covered below, with code you can paste.

Calling a local model with the ollama package

Install Ollama, pull a model, and make sure the server is running. Then install the Python client:

Copypip install ollama
ollama pull llama3.2

A basic chat call looks like this. The model name is whatever you pulled; llama3.2 is only an example.

Copyimport ollama

response = ollama.chat(
    model="llama3.2",
    messages=[{"role": "user", "content": "Explain DNS in two sentences."}],
)
print(response["message"]["content"])

Streaming

Pass stream=True and you get an iterator of chunks instead of one reply. Each chunk carries a piece of the answer in message.content.

Copyimport ollama

stream = ollama.chat(
    model="llama3.2",
    messages=[{"role": "user", "content": "Write a haiku about servers."}],
    stream=True,
)
for chunk in stream:
    print(chunk["message"]["content"], end="", flush=True)
print()

Pointing the client at another machine

If Ollama runs on a different host, create a client with an explicit address, for example ollama.Client(host="http://192.168.1.20:11434"). By default the server only listens on the local machine, so remote access needs a deliberate setting on the server side. Do not expose it to the public internet without a proxy and authentication in front, because the native API has no key check of its own.

Plain HTTP with requests

You do not need the package at all. The native chat endpoint is POST /api/chat. Streaming is the default there, and the stream is newline-delimited JSON, not SSE. Set "stream": false if you want a single object back.

Copyimport requests

r = requests.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "llama3.2",
        "messages": [{"role": "user", "content": "Name three HTTP status codes."}],
        "stream": False,
    },
    timeout=120,
)
r.raise_for_status()
print(r.json()["message"]["content"])

Set a generous timeout. The first request after a pull has to load the model into memory, and that can take far longer than later requests.

Reading the stream by hand is only a few more lines. Call requests.post(..., stream=True), loop over r.iter_lines(), skip empty lines, and run json.loads on each one. Every object holds a small piece of the reply, and the last one has "done": true. This is handy in a minimal container where you would rather not install another package, and it makes clear what the library does for you. The hosted API streams differently, as SSE lines that begin with data:, which is one more reason to let the OpenAI SDK parse the stream for you in portable code.

Using the OpenAI SDK with Ollama

Ollama also exposes an OpenAI-style endpoint under /v1. That means the official openai package works, and so does most software that lets you set a custom base URL. The key is required by the SDK but ignored by a local Ollama, so any string will do.

Copyfrom openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
resp = client.chat.completions.create(
    model="llama3.2",
    messages=[{"role": "user", "content": "Give me a one-line summary of TCP."}],
)
print(resp.choices[0].message.content)

This is the most useful form, because it decouples your code from the server. The same function can talk to a local model today and to something else tomorrow, which is the next section.

The same code against a hosted API

The hosted option here is an OpenAI-compatible API with a single model, uncensored, run by an independent provider and tuned to answer without content refusals for lawful adult use. It is text only. The base URL is https://api.ollamaapi.cc/v1, and you authenticate with a Bearer key from the API key page.

Here is a version that reads its target from environment variables, with the hosted values as defaults. To go back to your local server, export two variables and change nothing else.

Copyimport os
from openai import OpenAI

client = OpenAI(
    base_url=os.getenv("LLM_BASE_URL", "https://api.ollamaapi.cc/v1"),
    api_key=os.getenv("LLM_API_KEY", "YOUR_API_KEY"),
)
MODEL = os.getenv("LLM_MODEL", "uncensored")

resp = client.chat.completions.create(
    model=MODEL,
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user", "content": "List four uses of a message queue."},
    ],
    temperature=0.7,
)
print(resp.choices[0].message.content)

For local runs, set LLM_BASE_URL=http://localhost:11434/v1, LLM_API_KEY=ollama and LLM_MODEL=llama3.2. The quickstart has the same setup in curl and Node.

Streaming, JSON mode and tools on the hosted side

The hosted endpoint supports streaming over SSE, JSON mode, tool calling, and the usual sampling parameters (temperature, top_p, stop, seed and the penalties). Streaming with the SDK is the same loop you would write for any OpenAI-style server:

Copystream = client.chat.completions.create(
    model="uncensored",
    messages=[{"role": "user", "content": "Tell a two-paragraph story about a lighthouse."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

For structured output, set response_format={"type": "json_object"} and say in the prompt that you want JSON. Always parse inside a try block anyway; a model can still return something you did not expect.

One difference from local use matters in code: limits. A full key allows 8 concurrent requests and 300 requests per minute, and a streamed request keeps its slot until the stream ends. If you fan out with threads, cap the pool at 8 and retry on HTTP 429 using the Retry-After header when it is present.

Keeping a conversation going

Neither the local nor the hosted chat endpoint remembers anything between calls. The server sees only the messages you send, so a chatbot is a list that you append to and resend. The pattern is identical for both targets, which is one more reason to keep the OpenAI-style client as your default.

Copyhistory = [{"role": "system", "content": "You are a helpful assistant."}]

def ask(text):
    history.append({"role": "user", "content": text})
    resp = client.chat.completions.create(model=MODEL, messages=history)
    answer = resp.choices[0].message.content
    history.append({"role": "assistant", "content": answer})
    return answer

print(ask("Pick a name for a hiking app."))
print(ask("Give me a tagline for it."))

Two practical notes. First, local models have a context length that you set and that memory has to hold; if the history grows past it, older turns are cut or the call fails, depending on settings. The hosted model accepts a 100,000-token window including the completion, which is far more than most chats need, but long histories still cost input tokens on every turn. Second, once a history gets long, summarise the oldest turns into one short message instead of resending them all. That keeps latency and cost steady on either backend.

Retrying politely

On the hosted API a temporary overload returns HTTP 429 with a Retry-After: 5 header. A small wrapper handles it:

Copyimport time
from openai import RateLimitError

def chat_with_retry(messages, tries=4):
    for attempt in range(tries):
        try:
            return client.chat.completions.create(model=MODEL, messages=messages)
        except RateLimitError:
            time.sleep(5 * (attempt + 1))
    raise RuntimeError("still rate limited")

Local Ollama does not rate limit you, but it queues requests when the model is busy, so the same wrapper does no harm there.

Local or hosted: how to decide

Neither choice is better in general. Use the points below as a checklist.

• Choose local Ollama if you need to work offline, if data must never leave the machine, if you already own a capable GPU, or if you want to try many different open models quickly.
• Choose a hosted API if you have no GPU, if you run on a small VPS or in CI, if several people or services share one backend, or if you want a long context window without sizing memory yourself. The hosted model accepts up to 100,000 tokens of prompt plus completion.
• Mix both during development: iterate locally, then point production at the hosted URL through environment variables.

Hosted use is billed per token, at $0.15 per 1M input tokens and $0.50 per 1M output tokens, from prepaid credit. The pricing page lists top-up amounts, and new accounts can claim a $0.50 trial with one click, no card needed. Prompts are not used for training.

Errors you will meet

Symptom	Likely cause	Fix
Connection refused on port 11434	The Ollama server is not running	Start the app or run ollama serve
Model not found	The model was never pulled, or the name has a typo	Run ollama list, then ollama pull the name you use
First call times out	The model is still loading	Raise the timeout, or send a warm-up request
Authentication error on the hosted API	Missing or wrong Bearer key	Check LLM_API_KEY and the base URL
HTTP 429 with code concurrency	More than 8 requests in flight (2 on a trial key)	Lower the worker count
HTTP 429 with code rate_limited	Over the requests-per-minute limit	Back off and retry

Failed or refused requests on the hosted side are not charged, so retry logic does not cost you for the failed attempts.

More guides

• Ollama API Pricing: Local Cost vs Pay per Token
• Ollama API vs OpenRouter: Which One Fits You
• Mistral API Key Guide