Robility Interaction Scope

Estimated reading: 5 minutes

Robility Interaction Scope is a container activity that identifies and manages the target desktop or web application, allowing you to perform multiple UI interactions within the same application scope. It uses AI-powered application indication and UI Explorer to identify the target application and its UI elements.

Note: Double-click the Robility Interaction Scope activity to open and configure its settings.

Menu Icon (Three line)

1. Change Informative Screenshot: Change the existing screenshot by selecting a new screenshot from your local system.
2. Remove Informative Screenshot: Remove the existing informative screenshot.
3. Indicate Target Application: The Indicate Target Application feature allows you to identify and select UI elements from desktop or web applications using UI Explorer.

UI Explorer

1. Click Spy to start identifying an application or UI element.
2. Select the required application or UI element from the desktop or web application.
3. After selecting the application, you are redirected to the Selection Helper tab.
4. To return to UI Explorer from the Selection Helper, click Indicate Application via UI Explorer.
5. The Window Selector is displayed automatically when the application is indicated.

A. To use the Window Selector as a variable, select Use Variable.
B. Enter a variable name and click Bind.
C. The variable name is then displayed in the Window Selector field.
D. You can delete the Window Selector if it is not required.

6. You can create a new Variable or Argument to store and reuse the selected application or UI element information.
7. If the current indication is incorrect or the application has changed, click Re-indicate to select the application again.
8. After completing the required configuration, click Confirm to save the selected application and its UI elements.

UI Explorer displays all currently open applications and web pages that can be selected for UI element identification.

a. Select an application from the list of open applications.
b. The compatible selectors for the selected application are displayed.
c. Select the required selector to identify and interact with the corresponding UI element.
d. Once done, click Set as Application to confirm the selection.

9. Open Application Instance: If the target application is already open, you can either select Focus Current Instance to bring the existing application instance to the foreground or select Open New Instance to launch a new instance.

Activity in the Canvas

Indicated Application: Displays the image of the application selected using Indicate Target Application.

File Path: Automatically displays the file path of the indicated application.

Sequence: Displays the sequence used to organize and execute the activities.

Robility ScreenPlay Task: Allows you to add multiple activities within the scope to perform the required UI interactions.

Properties

AI Connection

API Key: Enter a valid API key to authenticate the AI connection.

Default Model: Enter the model name. Gemini 3.5 Flash is recommended.

MISC

Display Name: Displays the name of the activity. You can customize the activity name to help troubleshoot issues more easily. The specified name is also used for logging purposes.

Version: Displays the version number of the activity.

Options

Delay After (ms): Specifies the delay after the activity is executed. The default value is 5000 ms.

Open Mode: Specifies how the target application should be opened. The available options are:
If Not Open: Opens the application only if it is not already open.
Always: Opens a new instance of the application.
Never: Does not open the application.

Target Application

Application Arguments: Specifies the arguments required to launch the target application. You can enter the arguments directly or provide a variable containing the argument values.

Application Path: Displays the path of the application automatically based on the indicated target application.

Target Selector: Displays the selector automatically based on the indicated target application.

URL: Displays the URL automatically based on the indicated target application.

Working Directory: Displays the working directory automatically based on the indicated target application.

Note: For Application Path, Target Selector, URL, and Working Directory, you can either use the value displayed automatically or provide a variable containing the required value.

How to Use the Robility Interaction Scope

1. Drag and drop the Robility Interaction Scope into the workflow, then double-click it to open the configuration panel.
2. Once the configuration window opens, you will see the Indicate Target Application option.
3. You can select the target application in one of the following ways:
a. Choose an application from the list of applications currently open on your machine.
b. Use the Spy option to identify and select the target application directly.

If you select an application from the list:

i. Click Set as Application.
ii. You will automatically be redirected to the Selection Helper tab.
iii. Click Confirm to complete the selection.

If you use the Spy option:

i. You will automatically be redirected to the Selection Helper tab.
ii. Click Confirm to complete the selection.

4. By default, a Robility Screenplay Task activity is added within the scope.
a. You can add additional Screenplay Task activities to the sequence if required.
b. In the activity, provide the appropriate prompt and configure the necessary properties.

5. Once the target application has been indicated and the prompt has been configured, run the workflow by clicking Run.
a. The workflow will be executed based on the selected application and the provided prompt.

Robility Interaction Scope

Menu Icon (Three line)

UI Explorer

Activity in the Canvas

Properties

AI Connection

MISC

Options

Target Application

How to Use the Robility Interaction Scope

Robility Interaction Scope

CONTENTS