Batch Run
The Batch Run component runs a language model over each row of one text column in a DataFrame and then returns a new DataFrame with the original text and an LLM response. The output contains the following columns:
1. text_input: The original text from the input DataFrame
2. model_response: The model’s response for each input
3. batch_index: The 0-indexed processing order for all rows in the DataFrame
4. metadata (optional): Additional information about the processing
Use the Batch Run component in a flow
If you pass the Batch Run output to a Parser component, you can use variables in the parsing template to reference these keys, such as {text_input} and {model_response}. This is demonstrated in the following example.
1. Connect a Language Model component to a Batch Run component’s Language model
2. Connect DataFrame output from another component to the Batch Run component’s DataFrame For example, you could connect a File component with a CSV file.
3. In the Batch Run component’s Column Name field, enter the name of the column in the incoming DataFrame that contains the text to process. For example, if you want to extract text from a name column in a CSV file, enter name in the Column Name
4. Connect the Batch Run component’s Batch Results output to a Parser component’s DataFrame
Optional: In the Batch Run component’s header menu, click Controls, enable the System Message parameter, click Close, and then enter an instruction for how you want the LLM to process each cell extracted from the file. For example, Create a business card for each name.
5. In the Parser component’s Template field, enter a template for processing the Batch Run component’s new DataFrame columns (text_input, model_response, and batch_index):
For example, this template uses three columns from the resulting, post-batch DataFrame:
record_number: {batch_index}, name: {text_input}, summary: {model_response}
6. To test the processing, click the Parser component, click Run component, and then click Inspect output to view the final DataFrame.
You can also connect a Chat Output component to the Parser component if you want to see the output in the Playground.
Batch Run parameters
Some Batch Run component input parameters are hidden by default in the visual editor. You can toggle parameters through the Controls in the component’s header menu.
| Name | Type | Description |
|---|---|---|
| model | HandleInput | Input parameter. Connect the 'Language Model' output from a Language Model component. Required. |
| system_message | MultilineInput | Input parameter. A multi-line system instruction for all rows in the DataFrame. |
| df | DataFrameInput | Input parameter. The DataFrame whose column is treated as text messages, as specified by 'column_name'. Required. |
| column_name | MessageTextInput | Input parameter. The name of the DataFrame column to treat as text messages. If empty, all columns are formatted in TOML. |
| output_column_name | MessageTextInput | Input parameter. Name of the column where the model's response is stored. Default=model_response. |
| enable_metadata | BoolInput | Input parameter. If True, add metadata to the output DataFrame. |
| batch_results | DataFrame | Output parameter. A DataFrame with all original columns plus the model's response column. |
