Ollama to Docker Model Runner: The Endpoint Map

I tested what it actually takes to move an existing Ollama client over to Docker Model Runner (DMR), and the answer splits three ways: projects on Ollama's native API barely change, projects on the OpenAI-compatible API must change one base URL, and projects that only use embeddings hit a trap nobody warns you about. The embeddings endpoint Docker is compatible with is the one Ollama's own docs have marked as superseded.

⚠️ What "tested" means here: this machine has no Docker Desktop and no usable GPU, so I could not run a real DMR server process. Every "tested" claim below is about the real HTTP request lines emitted by three official clients — openai 3.32.0, ollama 0.6.3 and httpx 0.28.1 — across 29 configurations. The receiving server is a local replay server I built from the endpoint table in Docker's api-reference page; it only answers the question "does this path land on a route Docker has published?" Real DMR server behavior follows the official docs, and anything the docs do not cover is flagged as such below.

I tested 29 configurations and 11 of them — 37.9% — landed on routes Docker publishes. Here is what that means for your migration.

Three cost tiers: count your lines before you start

The most common mistake is assuming a port swap is enough. The real cost depends entirely on which API layer you sit on.

The third tier is where migrations actually stall. ollama create -f Modelfile has no HTTP equivalent in DMR; Docker's replacement is the CLI's docker model package --gguf. The embeddings mismatch is subtler, and it gets its own section below.

Map 1 — OpenAI-compatible layer: `/v1` becomes `/engines/v1`

Both sides implement an OpenAI-compatible API, but the path prefixes differ completely, and that is where most 404s come from.

Two details matter in that right-hand column. First, the "single model" template is two segments, {namespace}/{name}, not one — official model names always carry the ai/ prefix, so the real path looks like /engines/v1/models/ai/smollm2. Second, Ollama documents /v1/responses and DMR does not; my test run sent POST /engines/v1/responses, which falls outside the published routes.

DMR also lets you put the engine name in the path, as /engines/llama.cpp/v1/chat/completions. That variant also landed on a published route in my run, and it only matters when you run several inference engines side by side.

One more documented difference worth a regression pass: Docker states that token counting "uses the model's native token encoder, which may differ from OpenAI's." If your code bills or truncates based on usage, check that path again.

Map 2 — Ollama-compatible layer: 5 against 15

Docker states plainly that the base URL for "Ollama-compatible clients" is http://localhost:12434, with no path at all. Counting from the compatibility table in the api-reference page, there are exactly five endpoints:

Ollama's own docs/api.md lists 15 endpoint paths: /api/generate, /api/chat, /api/create, /api/tags, /api/show, /api/copy, /api/delete, /api/pull, /api/push, /api/embed, /api/embeddings, /api/ps, /api/version, plus /api/blobs/{sha256} (one HEAD to check existence, one POST to push a blob). By the docs, that is 5 out of 15, roughly 33% coverage.

The mismatch that matters most: Ollama's docs say about /api/embeddings, in these words, this endpoint has been superseded by /api/embed. Ollama has moved on to /api/embed. DMR's compatibility list contains the old /api/embeddings and not /api/embed. So a project written to Ollama's current best practice will 404 after migration — my test run of the ollama SDK's embed() sent POST /api/embed, outside the published routes.

The remaining gaps have no documented HTTP replacement either: /api/pull and /api/push map to the CLI's docker model pull and docker model push; /api/create (Modelfile) maps to docker model package; /api/ps is missing from the HTTP compatibility layer but exists as the CLI's docker model ps; /api/version maps to docker model version.

Map 3 — the CLI

Docker's CLI reference lists more than 30 subcommands under docker model, a wider management surface than ollama's (df, bench, status, purge, unload, gateway and more). If your reason for migrating is "I want something that behaves like the rest of my container workflow," this layer is a genuine win.

Map 4 — model names and environment

Smallest change, easiest to miss.

• Namespace. Ollama documents model:tag with an optional namespace, giving orca-mini:3b-q8_0 as an example. DMR requires "the full model identifier including the namespace" and gives ai/smollm2, ai/llama3.2. So smollm2 becomes ai/smollm2.
• Quant tag casing. Ollama's examples use lowercase q8_0; DMR's examples use uppercase Q4_K_M (as in ai/smollm2:360M-Q4_K_M). I could not verify on this machine whether the DMR server is case-sensitive, but since that is how the official examples are written, copy the uppercase form.
• Auth. Ollama's OpenAI compatibility doc says api_key='ollama', # required but ignored. DMR says "Not required. DMR ignores the Authorization header." Both ignore the key, but the client usually still requires a non-empty value, and leaving it out fails on the SDK side.
• Port. Ollama defaults to 11434; DMR defaults to 12434. On Docker Desktop that TCP port is off by default — enable host-side TCP support in the AI settings tab and set the port. On Docker Engine, the docs state "TCP support is enabled by default for Docker Engine on port 12434".
• From inside a container. The docs give three different answers: http://model-runner.docker.internal with no port inside Desktop containers, http://172.17.0.1:12434 for Engine containers, and possibly extra_hosts: "model-runner.docker.internal:host-gateway" inside a Compose project. I could not verify any of these here; the inconsistency itself is unpacked in the previous article on Compose integration.

What the 29 test runs actually produced

Two conclusions you can act on immediately: do not obsess over the trailing slash — the openai SDK produced an identical path with and without it; and never put a path in the ollama SDK's host, because it appends /api/xxx to whatever path you gave it.

To reproduce this yourself, start a server that records request lines and fire both SDKs at it:

import threading, http.server, socketserver
from openai import OpenAI

SEEN = []

class H(http.server.BaseHTTPRequestHandler):
    def do_POST(self):
        n = int(self.headers.get("Content-Length") or 0)
        self.rfile.read(n)
        SEEN.append(self.path)          # the real request line
        body = b'{"choices":[{"message":{"role":"assistant","content":"ok"}}]}'
        self.send_response(200)
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

srv = socketserver.TCPServer(("127.0.0.1", 18888), H)
threading.Thread(target=srv.serve_forever, daemon=True).start()

OpenAI(base_url="http://127.0.0.1:18888/engines/v1", api_key="x", max_retries=0) \
    .chat.completions.create(model="ai/smollm2", messages=[{"role": "user", "content": "hi"}])
print(SEEN)   # ['/engines/v1/chat/completions']

Troubleshooting: 5 real errors

All five were reproduced on this machine. The request lines and status codes are measured; the response bodies quoted below come from my replay server, so a real DMR 404 may look different. The exception types come from the official SDKs.

Error 1 — port swap only: `openai.NotFoundError: Error code: 404`

openai.NotFoundError: Error code: 404 - {'error': {'message': 'no route for /v1/chat/completions'}}

The measured request line was POST /v1/chat/completions. Cause: you kept Ollama's documented base_url='http://localhost:11434/v1/' and only changed the port. The fix replaces the whole base URL:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:12434/engines/v1", api_key="not-needed")

Error 2 — a path in the ollama SDK host: `ollama.ResponseError`

ollama.ResponseError: no route for /engines/v1/api/chat

Measured request line POST /engines/v1/api/chat. Client(host=...) keeps the path you give it and appends /api/chat. Fix: end the host at the port, Client(host="http://localhost:12434"). For the same reason, Client(host="http://localhost:12434/v1") produces POST /v1/api/chat and fails identically.

Error 3 — embeddings mismatch: `ollama.ResponseError: no route for /api/embed`

ollama.ResponseError: no route for /api/embed

Measured request line POST /api/embed. This is the most counter-intuitive one: your code follows Ollama's current documentation exactly, but DMR's compatibility list only has the old /api/embeddings. Two ways out — stay on the Ollama layer and call /api/embeddings (note its parameter is prompt, not /api/embed's input), or move to the OpenAI layer and use /engines/v1/embeddings, which landed on a published route in testing.

Error 4 — no Responses API: `openai.NotFoundError: Error code: 404`

openai.NotFoundError: Error code: 404 - no route for /engines/v1/responses

Measured request line POST /engines/v1/responses. Ollama's openai-compatibility doc lists /v1/responses; DMR's endpoint table does not. Switch back to chat.completions.create(...).

Error 5 — single-segment model name misses the path template

My test sent GET /engines/v1/models/smollm2 for models.retrieve("smollm2"), while DMR's documented template is two segments, /engines/v1/models/{namespace}/{name}. Fix: name the model ai/smollm2. I could not verify the failure shape against a real DMR instance (my replay server accepts single-segment names), so treat this one as derived from the documented template, not measured server behavior.

When to hold off

• You depend on Modelfile. ollama create -f Modelfile has no HTTP equivalent. The official replacement, docker model package --gguf, packages a GGUF file as an OCI Artifact, and your template, system prompt and parameters move elsewhere (docker model configure).
• You rewrote embeddings onto the new endpoint. See error 3: you would be migrating onto the endpoint Ollama itself marks superseded.
• Your dashboard relies on /api/ps. No HTTP equivalent; move to the CLI's docker model ps.
• You run Docker Desktop on Windows. The docs are explicit that host-side TCP must be enabled or nothing listens on 12434, and GPU-backed inference is a separate toggle.

FAQ

Q: If I only change 11434 to 12434, what still works?

A: In my testing, /api/chat, /api/tags, /api/generate, /api/show and /api/embeddings all land on routes Docker publishes. The conditions are that you stay on the Ollama-compatible layer and that your model names carry a namespace such as ai/.

Q: Which base URL do I use for the OpenAI layer?

A: Docker's "Base URLs for third-party tools" table splits three ways: OpenAI SDKs use http://localhost:12434/engines/v1; Anthropic SDKs use http://localhost:12434; Ollama-compatible clients also use http://localhost:12434. The difference is entirely the /engines/v1 suffix.

Q: Which embeddings endpoint should I use?

A: On the Ollama layer, /api/embeddings (parameter prompt). On the OpenAI layer, /engines/v1/embeddings. Ollama's current /api/embed has no counterpart in DMR's docs — do not use it.

Q: How do I connect from inside a container?

A: The docs give three answers: http://model-runner.docker.internal with no port for Desktop, http://172.17.0.1:12434 for Engine, and possibly extra_hosts for Compose projects. I could not run a real environment here, so pick based on Desktop versus Engine and watch requests with docker model logs.

Q: How do I check my DMR version?

A: The get-started page gives docker model version. This machine has no Docker, so I cannot quote a version number; everything here about DMR comes from Docker's docs as captured on 2026-10-17. Trust your local docker model version output.

Related reading

• Docker Model Runner architecture: three engines and two address conventions
• Model Runner with Open WebUI: 4 real failures
• Declaring models in Compose: endpoint injection and 4 real errors

Every official claim here comes from Docker's api-reference, get-started, configuration and CLI reference pages, plus Ollama's docs/api.md and docs/api/openai-compatibility.mdx, captured 2026-10-17. Ollama's latest release at that time was v0.40.5, published 2026-10-14. Versions and ports are stated as of publication — confirm against the official pages before you rely on them.

👉 Join Xiaomi MiMo Platform: Leading AI model platform with cost-effective inference

👉 Join Aliyun AI: Top AI products with exclusive coupons for business innovation

📌 This article was AI-assisted generated and human-reviewed | TechPassive — An AI-driven content testing site focused on real tool reviews