DataFrame Operations

Estimated reading: 3 minutes

The DataFrame Operations component performs operations on DataFrame (table) rows and columns, including schema changes, record changes, sorting, and filtering. For all options, see DataFrame Operations parameters.

The output is a new DataFrame containing the modified data after running the selected operation.

Use the DataFrame Operations component in a flow

The following steps explain how to configure a DataFrame Operations component in a flow. You can follow along with an example or use your own flow. The only requirement is that the preceding component must create DataFrame output that you can pass to the DataFrame Operations component.

1. Create a new flow or use an existing flow.

API response extraction flow example

The following example flow uses five components to extract Data from an API response, transform it to a DataFrame, and then perform further processing on the tabular data using a DataFrame Operations component. The sixth component, Chat Output, is optional in this example. It only serves as a convenient way for you to view the final output in the Playground, rather than inspecting the component logs.

If you want to use this example to test the DataFrame Operations component, do the following:

i. Create a flow with the following components:

a. API Request
b. Language Model
c. Smart Function
d. Type Convert

ii. Configure the Smart Function component and its dependencies:

a. API Request: Configure the API Request component to get JSON data from an endpoint of your choice and then connect the API Response output to the Smart Function component’s Data input.
b. Language Model: Select your preferred provider and model and then enter a valid API key. Change the output to Language Model and then connect the LanguageModel output to the Smart Function component’s Language Model input.
c. Smart Function: In the Instructions field, enter natural language instructions to extract data from the API response. Your instructions depend on the response content and desired outcome. For example, if the response contains a large result field, you might provide instructions like exploding the result field out into a Data object.

iii. Convert the Smart Function component’s Data output to DataFrame:

a. Connect the Filtered Data output to the Type Convert component’s Data input.
b. Set the Type Convert component’s Output Type to DataFrame.

Now the flow is ready for you to add the DataFrame Operations component.

2. Add a DataFrame Operations component to the flow and then connect DataFrame output from another component to the DataFrame input.

All operations in the DataFrame Operations component require at least one DataFrame input from another component. If a component doesn’t produce DataFrame output, you can use another component, such as the Type Convert component, to re-format the data before passing it to the DataFrame Operations component. Alternatively, you could consider using a component that is designed to process the original data type, such as the Parser or Data Operations components.

If you are following along with the example flow, connect the Type Convert component’s DataFrame Output port to the DataFrame input.

3. In the Operations field, select the operation you want to perform on the incoming DataFrame. For example, the Filter operation filters the rows based on a specified column and value.

Tip You can select only one operation. If you need to perform multiple operations on the data, you can chain multiple DataFrame Operations components together to execute each operation in sequence. For more complex multi-step operations, like dramatic schema changes or pivots, consider using an LLM-powered component, like the Structured Output or Smart Function component, as a replacement or preparation for the DataFrame Operations component

DataFrame Operations

Use the DataFrame Operations component in a flow

API response extraction flow example

DataFrame Operations

CONTENTS