LLM Configuration Management
LLM Configuration Management in Robility provides a centralized governance framework to configure, manage, and control access to LLM providers, enabling users and automations to consume configured language models within Robility Manager across the tenant.
Tenant administrators have exclusive permission to manage and modify this configuration. It enables integration with multiple providers by allowing configuration of provider-specific settings and common advanced parameters, ensuring controlled access, consistent behavior, and compliance across all configured providers.
The configured details can be utilized through the Robility Model component within a workflow, which retrieves the configuration dynamically at runtime, eliminating the need to hardcode provider settings or credentials.
Purpose
1. Centralize the configuration of LLM endpoints, API keys, and deployments.
2. Securely store and manage authentication credentials in an encrypted manner. All credentials are stored in the vault configured in the Vault Settings section, enabling seamless injections into automated workflows without hardcoding sensitive data into scripts or configuration files.
3. Enable seamless integration of AI models into workflows.
4. Ensure that updates to provider configurations are applied instantly across all dependent workflows.
Provider Configuration
The Provider Configuration section defines the connection and access details for the selected provider. Fields in the Basic Configuration and Authentication sections vary depending on the provider selected.
1. Basic Configuration
This section defines the core setup needed to identify and connect to the selected provider. The configured values are used to route requests and invoke the appropriate model.
2. Authentication
This section is used to configure the access credentials required by the selected provider. Credentials are validated during request execution to ensure only authorized requests are processed.
All authentication credentials are securely stored in a Key Vault within the Settings page. Use the Robility Model component to retrieve these secrets at runtime during request execution.
3. Advanced Settings
Advanced Settings are common across all providers and are used to control model behavior, request handling, and compliance requirements.
a. Model Parameters
Model Parameters define how the language model generates responses.
1. Temperature — Controls the randomness of the generated response. Defaults to 0.7.
Higher values (0.7 – 1.0): Produce more varied and creative responses.
Lower values (0.0 – 0.6): Result in more consistent and predictable outputs.
2. Max Tokens — Defines the maximum length of the generated response. Once the limit is reached, response generation stops. Defaults to 2048 tokens.
Higher values (2048 – 4096): Allow longer responses but may increase costs or hit model limits.
Lower values (1 – 2047): Reduce costs but may truncate the input or output.
3. Top-p — Controls the diversity of the generated response by limiting the probability distribution of possible tokens. Defaults to 1.
Higher values (0.8 – 1.0): Increase variation and diversity in the response.
Lower values (0.1 – 0.7): Produce more focused and predictable outputs.
b. Network Settings
Network Settings define how requests are routed and managed when communicating with the provider.
1. Timeout (seconds) – Specifies the maximum duration the system waits for a response. The default value is 60 seconds, after which the request is terminated if no response is received. This value can be increased to accommodate longer processing times.
2. Retry Counts – Defines the number of retry attempts in case of request failure. The default value is 3, after which no further attempts are made. This value can be increased if needed.
c. Enterprise Compliance
These settings ensure data handling meets organizational and regulatory requirements.
1. Data Residency Region – Specifies where all data is stored and processed. Once a region is selected, all data remains within that location to meet data privacy and compliance requirements.
2. PII Reduction Enabled – When enabled, personal information such as names, emails, and phone numbers are automatically detected and masked from prompts and responses before processing or storage.
3. Auto Logging Enabled – When enabled, all LLM requests and responses are automatically captured and stored in logs for monitoring, debugging, and auditing.
4. Audit Retention Days – The number of days audit logs and interaction records are kept. After the specified period, the data is permanently purged. Defaults to 90 days.
Configuration Actions
After adding all required details, click Validate. Once validation is successful, click the View option in the added provider row to see the model details, and select Edit to make changes.
Only users with the Tenant Admin role can view and edit the LLM configuration. API keys are masked and accessible only to authorized roles.
