View Source Instructor.Adapters.VLLM (Instructor v0.1.0)

VLLM adapter for Instructor.

Configuration

config :instructor, adapter: Instructor.Adapters.VLLM, vllm: [
  api_url: "http://localhost:8000" # (Optional) defaults to localhost:8000
]copy

or at runtime:

Instructor.chat_completion(..., [
  adapter: Instructor.Adapters.VLLM,
  api_url: "http://localhost:8000" # (Optional) defaults to localhost:8000
])copy

To get a VLLM API key, see VLLM.