INTEGRATIONS / LLM PROVIDERS

LLM providers

promptLM routes prompt execution through Spring AI by default. Two providers ship with the CLI distribution: OpenAI and Anthropic. For everything else, enable the LiteLLM gateway module, which fronts a long tail of providers behind an OpenAI-compatible API.

Default providers (Spring AI)

The CLI’s application.yml wires Spring AI’s OpenAI and Anthropic clients to the standard environment variables:

Spring property Reads from

spring.ai.openai.api-key

OPENAI_API_KEY

spring.ai.anthropic.api-key

ANTHROPIC_API_KEY

Routing between OpenAI and Anthropic is driven by the vendor and model fields on the spec’s request — see Request types.

LiteLLM gateway (opt-in)

The promptlm-execution-litellm module fronts LiteLLM as a self-managed Docker container and routes prompts through it as an OpenAI-compatible proxy. Disabled by default.

Enable with:

export PROMPTLM_GATEWAY_LITELLM_ENABLED=true

Or in application.yml:

promptlm:
  gateway:
    litellm:
      enabled: true

Properties

All keys are nested under promptlm.gateway.litellm.

Property Default Purpose

enabled

false

Master switch for the LiteLLM gateway.

version

v1.82.3.dev.6

LiteLLM Docker image tag. Used to compose ghcr.io/berriai/litellm:<version> unless docker.image is set.

base-url

http://localhost:4000

Base URL where LiteLLM is exposed.

vendor

litellm

Prompt vendor identifier this gateway handles.

model-aliases

empty map

Mapping from promptLM model id to LiteLLM route.

environment

empty map

Extra environment variables passed to the LiteLLM container.

Docker container management

Nested under promptlm.gateway.litellm.docker:

Property Default Purpose

manage

true

Whether promptLM manages the container’s lifecycle.

image

(derived from version)

Explicit image reference. Must use an immutable tag or digest — no :latest.

container-name

promptlm-litellm

Container name when promptLM spawns LiteLLM.

port

4000

Host port to expose.

network

(none)

Optional Docker network to attach to.

readiness-timeout

30s

How long to wait for LiteLLM readiness.

readiness-path

/health

Path queried for readiness checks.

Model discovery

Nested under promptlm.gateway.litellm.discovery:

Property Default Purpose

enabled

false

Pull the model list from LiteLLM at runtime.

cache-ttl

5m

How long the discovered list is cached.

models-path

/v1/models

Path queried to list available models.