Login
Updated on November 3, 2026

LLM Connections

Estimated reading: 8 minutes

LLM Connections define how Control Plane authenticates with and communicates with Large Language Model (LLM) providers. Each connection stores the configuration required to send requests to a provider and access its supported models.

LLM Connections provide a reusable configuration that allows Control Plane to connect to supported LLMs without requiring connection details to be configured separately for each use case. These connections are used by the Playground feature to interact with configured models and experiment with their responses.

Why LLM Connections Are Needed

LLM Connections help you:

a. Connect to LLM providers: Configure the endpoint and authentication required to communicate with a provider.
b. Make models available: Enable default models or specify custom models supported by the configured endpoint.
c. Customize requests: Configure optional HTTP headers and provider-specific settings when required.
d. Experiment with models: Use configured LLM connections to interact with supported models in Playground.

Supported LLM Integrations

Control Plane supports the following LLM adapters for configuring connections:

a. Anthropic
b. OpenAI
c. Azure
d. Amazon Bedrock
e. Google Vertex AI
f. Google AI Studio

Only the LLM providers listed above can be added as LLM Connections in Control Plane. The available configuration fields and authentication methods depend on the selected adapter.

Some adapters support custom API endpoints compatible with their respective APIs. However, configuring a custom endpoint does not enable adding an LLM provider that is not included in the supported adapter list.

Navigation

To configure and manage LLM connections, navigate to Settings > LLM Connections in Control Plane.

LLM Connections provide the provider and model configurations required by the following feature:

Playground: Uses configured LLM connections to let users interact with supported models and experiment with responses.

For instructions on using Playground, refer to the respective documentation.

Connection Configuration

To create an LLM connection, navigate to Settings > LLM Connections and select the required LLM adapter. Enter the configuration details for the selected provider. The available fields depend on the adapter and its authentication method.

Provider-Specific Configuration

Anthropic and OpenAI

Anthropic and OpenAI use the same configuration fields.

Field Description
Provider Name* Specifies a unique identifier for the connection within Control Plane. The name cannot contain colons (:).
API Key* Specifies the API key used to authenticate requests to the selected provider.
API Base URL Specifies an optional custom endpoint for a provider-compatible API. Leave this field blank to use the provider's default endpoint. Use a custom endpoint when connecting through a supported gateway, proxy, or compatible service.
Extra Headers Specifies optional HTTP headers included with requests sent to the provider.
Enable Default Models Makes the provider's default models available within Control Plane. This option is enabled by default.
Custom Models Specifies one or more model names accepted by the configured provider endpoint.

Configuring Extra Headers

To add custom HTTP headers:

1. Select Add Header.
2. Enter the Header Name.
3. Enter the corresponding Header Value.
4. Repeat these steps to add additional headers, if required.

For example:
Header Name: X-Api-Version
Header Value: 2026-01-01

Note: Custom model names must exactly match the model names supported by the configured provider endpoint. Examples include gpt-4o, gpt-4o-mini, claude-sonnet-4-0, and claude-opus-4.

Azure

The following fields are available when Azure is selected as the LLM adapter.

Field Description
Provider Name* Specifies a unique identifier for the connection within Control Plane. The name cannot contain colons (:).
API Key* Specifies the Azure API key used to authenticate requests.
API Base URL* Specifies the Azure deployment endpoint. Enter the endpoint in the following format:

https://<instance-name>.openai.azure.com/openai/deployments
Extra Headers Specifies optional HTTP headers included with requests sent to Azure. Use this field when additional headers are required for authentication, routing, or other provider-specific configurations.
Custom Models Specifies one or more Azure deployment names accepted by the configured endpoint.
Note: Each custom model name must exactly match the deployment name configured in Azure. For example, my-gpt4-deployment.

Amazon Bedrock

The following fields are available when Amazon Bedrock is selected as the LLM adapter.

Field Description
Provider Name* Specifies a unique identifier for the connection within Control Plane. The name cannot contain colons (:).
Authentication Method Specifies the authentication method used to connect to Amazon Bedrock. The supported methods are AWS Access Keys and API Key.
AWS Region* Specifies the AWS Region hosting the Amazon Bedrock service. For example, us-east-1 or eu-west-1.
AWS Access Key ID Specifies the AWS access key ID used for authentication. This field is available only when AWS Access Keys is selected.
AWS Secret Access Key Specifies the AWS secret access key associated with the AWS Access Key ID. This field is available only when AWS Access Keys is selected.
Bedrock API Key* Specifies the API key used to authenticate requests to Amazon Bedrock. This field is available only when API Key is selected.
Custom Models* Specifies one or more Bedrock Inference Profile IDs accepted by the configured endpoint.

AWS Credentials

For self-hosted deployments, AWS credentials can be omitted when the environment provides credentials through the AWS SDK default credential provider chain.

When explicit AWS credentials are not provided, the system checks the available credential sources in the following order:

1. Environment variables, such as AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
2. The AWS credentials file, typically ~/.aws/credentials.
3. IAM roles for Amazon EC2 instances.
4. IAM roles for Amazon ECS tasks.

Example custom model: eu.anthropic.claude-sonnet-4-6

Note: AWS Region and Custom Models are required for the documented Amazon Bedrock configurations. AWS Access Key ID and AWS Secret Access Key are optional when the default credential provider chain supplies the credentials in a supported self-hosted environment.

Google Vertex AI

The following fields are available when Google Vertex AI is selected as the LLM adapter.

Field Description
Provider Name* Specifies a unique identifier for the connection within Control Plane. The name cannot contain colons (:).
Use Application Default Credentials (ADC) Enables authentication using credentials automatically available in the environment where Control Plane is running.
GCP Service Account Key (JSON) Specifies a Google Cloud service account key in JSON format. The service account must have the permissions required to access Vertex AI.
API Base URL Specifies an optional custom Vertex AI endpoint. Leave this field blank to use the default endpoint for the selected adapter.
Location (Optional) Specifies the Google Cloud region used for requests. Examples include global, us-central1, and europe-west4.
Enable Default Models Makes the default models for the selected Vertex AI adapter available within Control Plane. This option is enabled by default.
Custom Models Specifies one or more model names accepted by the configured Vertex AI endpoint.

Authentication Configuration

When Use Application Default Credentials (ADC) is enabled, the connection uses credentials available in the hosting environment, such as a service account associated with Google Compute Engine, Google Kubernetes Engine (GKE), or Cloud Run. A service account key does not need to be provided manually.

When ADC is disabled, provide a valid GCP Service Account Key (JSON) for the service account to be used.

Note: The global location is the default and is required for Gemini 3 models.

Examples of custom models include: gemini-2.5-pro, gemini-2.5-flash, text-embedding-005

Google AI Studio

The following fields are available when Google AI Studio is selected as the LLM adapter.

Field Description
Provider Name* Specifies a unique identifier for the connection within Control Plane. The name cannot contain colons (:).
API Key* Specifies the Google AI Studio API key used to authenticate requests.
API Base URL Specifies an optional custom endpoint for Google AI Studio-compatible APIs. Leave this field blank to use the default endpoint. A custom endpoint can be used with a supported gateway, proxy, or compatible service.
Enable Default Models Makes the default models for the selected adapter available within Control Plane. This option is enabled by default.
Custom Models Specifies one or more model names accepted by the configured endpoint.
Note: Custom model names must exactly match the model names supported by the configured Google AI Studio endpoint.

Examples include: gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-lite

Additional Notes

1. For Google Vertex AI, GCP Service Account Key (JSON) is required when ADC is disabled. When ADC is enabled, the hosting environment must provide valid credentials with the necessary Vertex AI permissions.
2. For Amazon Bedrock using AWS Access Keys, AWS Access Key ID and AWS Secret Access Key can be omitted when the supported AWS SDK default credential provider chain supplies the credentials.
3. For adapters that support default models, enabling Enable Default Models makes the provider’s default model options available. Custom models can be specified when additional models or deployment names are needed.
4. Custom model names, deployment names, and inference profile IDs must match the identifiers accepted by the configured provider endpoint.

Using LLM Connections

After configuring a connection, use the available LLM connection and model options in Playground to interact with the configured provider.

The connection determines how requests are authenticated and routed to the provider. The selected model determines which supported model or deployment processes the request.

For instructions on interacting with models, refer to the Playground documentation.

Project Settings - Previous API Keys Next - Project Settings Model Definitions
Share this Doc

LLM Connections

Or copy link

CONTENTS

Powered By Sutherland

Robility Chatbot
Robility Assistant
Online