Memory

Estimated reading: 7 minutes

Memory enables Robility Flow to retain conversation context and retrieve relevant information across interactions. Messages are stored in a configured Memory Base, and relevant records are made available to AI components in subsequent interactions.

Memory uses an embedding model and a vector database to store and retrieve information based on semantic similarity. When a new interaction begins, Robility Flow converts the input to a vector, searches the Memory Base for records with similar meaning, and passes the matching records to AI components as context.

With Memory, you can:

a. Continue a conversation across multiple interactions or sessions.
b. Provide AI components with relevant prior context.
c. Maintain consistent responses over time.

Why use Memory?

Large language models (LLMs) are stateless. They do not retain information between sessions, and they are limited by fixed context windows. Memory addresses these limitations by enabling your flows to:

a. Maintain cross-session continuity: Retain user preferences, past decisions, and historical context across separate runs and sessions.
b. Reduce token usage: Include only the most relevant past records in the prompt, rather than an entire, ever-growing conversation history.
c. Improve response accuracy: Surface relevant background details automatically, so users don’t need to repeat themselves.

Memory vs. chat history vs. knowledge base

Feature Memory Chat History Knowledge Base
Primary purpose Retains relevant conversational context for future AI reasoning Logs complete, chronological transcripts of conversations Stores static reference documents and domain knowledge
Stored content User prompts and AI responses (raw or preprocessed) Full, unedited message threads Uploaded files, manuals, FAQs, and external data
Retrieval method Semantic similarity search (vector embeddings) Sequential, session-based lookup Semantic and keyword search across indexed documents
Best used for Personalizing responses and recalling past user interactions Auditing conversations or displaying chat UI history Answering factual questions grounded in company data

How Memory works

Memory operates through two pipelines: Storage (Ingestion) and Retrieval.

Figure 1. Robility Flow Memory architecture: storage and retrieval flows

1. Memory storage (Ingestion)

Memory automatically captures user inputs and AI-generated responses from Playground sessions, API runs, Chat Input and Chat Output components, and Agent components.

a. Collection and batching: Interactions are grouped into conversation pairs (one user message and its corresponding AI response). Pairs remain in Pending status until the number of accumulated pairs reaches the configured Batch Size. They are then processed together as a batch.

b. LLM preprocessing (optional): When a batch is ready, Robility Flow formats the data according to the preprocessing setting:
i. Disabled (default): Each user message and AI response is embedded and stored as an individual record (Sender: User or Sender: Machine). Use this option when you need verbatim recall of exact exchanges.
ii. Enabled: The configured Preprocessing Model summarizes the batch into a single consolidated Memory Record before embedding. Use this option to reduce noise and store concise takeaways.

c. Embedding and vector storage: The configured Embedding Model converts the resulting text into vector embeddings and writes them to the vector database.

2. Memory retrieval

a. When a user sends a new prompt, the Memory Base component converts the input into a vector query.
b. The component runs a semantic search against the vector database. The search is scoped to either the active session or all sessions, depending on the Filter by Session setting.
c. The most relevant Memory Records are added to the AI component’s prompt as context, so the response is informed by prior interactions.

Getting started and configuration

Prerequisites

Before creating a Memory Base, ensure the following are active and accessible:

i. A configured DB Provider 
ii. An available Embedding Model

Accessing the Memories page

Log in to Robility Flow, open a flow in the Designer Canvas, and select the Memory icon (half-brain) in the right panel. The Memories page opens and lists your Memory Bases.

Figure 2. Memories page showing a Memory Base, its summary cards, and stored records

Configuration parameters

For step-by-step instructions, see Create a Memory Base.

Parameter Description
Memory Base Name A unique identifier for the Memory Base.
DB Provider The database provider used to store memory records.
Embedding Model The model used to generate vector embeddings for semantic search.
Batch Size The number of conversation pairs (one user message and one AI response) that must accumulate before processing is triggered. For example, 1 processes every exchange immediately, while 5 waits until five exchanges have accumulated.
LLM Preprocessing Determines how conversation batches are stored. When enabled, batches are summarized and consolidated before storage. When disabled, messages are stored individually in their original form.
Preprocessing Model The LLM used to process conversation batches when LLM Preprocessing is enabled.

Manage and monitor Memory Bases

Activation status and statistics

a. Activate toggle: A Memory Base must be activated to ingest new messages. Turning the Activate toggle off pauses processing, and incoming messages remain in Pending status until the Memory Base is reactivated.

b. Summary cards: The cards at the top of the page show the current state of the Memory Base:
i. Processed Messages: The number of messages successfully processed and stored.
ii. Pending Messages: The number of messages awaiting batch completion or activation.
iii. Last Generated: When Memory Records were last generated.
iv. Config: The configuration of the Memory Base, including the Embedding Model in use.
v. Chunk count: The label above the records table (for example, “Memory Base · 28 chunks”) shows the total number of stored chunks, which are the Memory Records.

View, filter, and search records

Each stored record shows its Sender (User or Machine), Job ID, Content, and Ingestion Timestamp. The user message and AI response from the same exchange share a Job ID.

a. Session filter: Use the dropdown in the top-right corner to view records for All Sessions or a specific session.
Note: The session filter changes only what is displayed in the UI. Retrieval scope at runtime is controlled by the Filter by Session setting on the Memory Base component.

b. Memory Base search: Use the search bar above the list of Memory Bases to find a Memory Base by name, for example, demomemory.

Delete a Memory Base

To delete a Memory Base, click the Delete (trash) icon. This permanently deletes the Memory Base and all of its records across all sessions, regardless of the current session filter.

Important: Individual Memory Records cannot be edited or deleted after they are stored.

Best practices and limitations

Best practices

1. Match vector dimensions: Verify that the index dimensions of your vector database match the output dimensions of the selected Embedding Model.
2. Tune Batch Size to traffic: Set Batch Size to 1 for real-time recall in low-volume conversations. Use higher values (5 or more) to improve throughput in high-volume flows.
3. Isolate multi-user context: In multi-user applications, enable Filter by Session on the Memory Base component and give each user a unique session ID, so that one user’s memories are not retrieved for another user.
4. Avoid storing sensitive data: Individual records cannot be selectively deleted, so do not route personally identifiable information (PII), credentials, or other sensitive data into Memory-enabled flows.

Known limitations

1. No granular deletion: Individual Memory Records cannot be edited or deleted. Only deleting the entire Memory Base is supported.

Troubleshooting

Issue Possible cause Resolution
Messages remain Pending The Memory Base is Inactive, or the Batch Size threshold has not been reached. Activate the Memory Base, or lower the Batch Size. For example, set it to 1 to process every exchange immediately.
No records appear after a run The Memory Base is Inactive, or the messages come from an unsupported source. Confirm that the Memory Base is Active and that messages originate from the Playground, an API run, Chat Input or Chat Output components, or an Agent component.
Database is unreachable The credentials are invalid, the connection settings are misconfigured, or the network is blocking the connection. Verify the DB Provider status, connection string, and credentials. Check that firewall rules allow the connection.
Embedding dimension mismatch The index dimensions of the vector database differ from the output dimensions of the Embedding Model. Reconfigure the vector database index, or select an embedding model with matching dimensions. If you change the embedding model after records are stored, you may need to create a new Memory Base, because existing vectors keep their original dimensions.
Wrong records returned at runtime The session scope of the Memory Base component does not match your use case. Check the Filter by Session setting in the Memory Base component. When it is enabled, retrieval is limited to the active session, so confirm that the session ID is the one you expect.
Share this Doc

Memory

Or copy link

CONTENTS
Robility Chatbot
Robility Assistant
Online