Action Chaining with ActionChainOptions

Action chaining is a powerful pattern in Brobot that allows you to compose multiple actions into complex workflows. The ActionChainOptions class provides a unified, type-safe way to chain actions together with different execution strategies.

Required Imports

All examples in this guide assume the following imports:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import io.github.jspinak.brobot.action.Action;
import io.github.jspinak.brobot.action.ActionChainOptions;
import io.github.jspinak.brobot.action.ActionResult;
import io.github.jspinak.brobot.action.ObjectCollection;
import io.github.jspinak.brobot.action.basic.click.ClickOptions;
import io.github.jspinak.brobot.action.basic.type.TypeOptions;
import io.github.jspinak.brobot.action.basic.find.PatternFindOptions;
import io.github.jspinak.brobot.action.internal.execution.ActionChainExecutor;
import io.github.jspinak.brobot.datatypes.state.stateObject.stateImage.StateImage;

Why Action Chaining?

Traditional automation often requires executing multiple actions in sequence:

• Click a button, then verify a menu appears
• Fill out a form with multiple fields
• Navigate through a multi-step wizard
• Perform complex drag-and-drop operations

ActionChainOptions provides a clean, fluent API for these scenarios while offering advanced features like nested searches and confirmation patterns.

Basic Concepts

Chaining Strategies

ActionChainOptions supports two chaining strategies:

1. NESTED: Each action searches within the results of the previous action

   • Useful for hierarchical UI navigation
   • Example: Find a dialog, then find a button within that dialog

2. CONFIRM: Each action validates the results of the previous action

   • Useful for reducing false positives
   • Example: Find potential matches, then confirm with additional criteria

Action Flow

When you chain actions:

1. The first action executes with the initial ObjectCollections
2. Subsequent actions can use results from previous actions (NESTED) or validate them (CONFIRM)
3. The chain continues until all actions complete or one fails
4. The final result contains the complete execution history

Two Ways to Execute Chains

Brobot provides two patterns for executing action chains:

1. Implicit Chaining (Recommended for Most Cases)

Use the then() method on any ActionConfig and execute with action.perform():

@Autowired
private Action action;

// Chain actions using then()
PatternFindOptions findAndClick = new PatternFindOptions.Builder()
    .setSimilarity(0.85)
    .then(new ClickOptions.Builder()
        .setPauseAfterEnd(0.5)
        .build())
    .build();

// Execute - chain runs automatically
ActionResult result = action.perform(findAndClick, buttonImage.asObjectCollection());

Advantages: Simpler, less boilerplate, no need to inject ActionChainExecutor.

2. Explicit Chaining (For Advanced Scenarios)

Use ActionChainOptions and ActionChainExecutor when you need fine control:

@Autowired
private ActionChainExecutor chainExecutor;

// Explicit chain configuration
ActionChainOptions chain = new ActionChainOptions.Builder(
    new ClickOptions.Builder().build())
    .setStrategy(ActionChainOptions.ChainingStrategy.NESTED)
    .then(new TypeOptions.Builder()
        .setTypeDelay(0.1)
        .build())
    .build();

// Execute with full control over parameters
ActionResult result = chainExecutor.executeChain(chain, new ActionResult(),
    buttonImage.asObjectCollection(),
    new ObjectCollection.Builder().withStrings("Hello World").build()
);

Advantages: Explicit strategy control, access to intermediate results, custom execution logic.

When to Use: Most users should start with implicit chaining. Use explicit chaining when you need custom strategy configuration or intermediate result access.

Simple Examples

Assumed Variables

The following examples assume you have:

• StateImage buttonImage - An initialized StateImage instance
• StateImage targetImage - Another StateImage for verification
• @Autowired Action action - Injected Brobot Action service

See the Production Examples section for complete, compilable code.

Sequential Actions

The most basic use case is executing actions in sequence.

Click and Verify

A common pattern is clicking something and verifying the result:

ActionChainOptions clickVerify = new ActionChainOptions.Builder(
    new ClickOptions.Builder()
        .setPauseAfterEnd(0.5)
        .build())
    .then(new PatternFindOptions.Builder()
        .setStrategy(PatternFindOptions.Strategy.FIRST)
        .setPauseBeforeBegin(2.0) // Wait for result
        .build())
    .build();

Advanced Patterns

Nested Search

Find elements within other elements using the NESTED strategy:

// Find a dialog, then find a button within it
ActionChainOptions nestedSearch = new ActionChainOptions.Builder(
    new PatternFindOptions.Builder()
        .setStrategy(PatternFindOptions.Strategy.FIRST)
        .build())
    .setStrategy(ActionChainOptions.ChainingStrategy.NESTED)
    .then(new PatternFindOptions.Builder()
        .setStrategy(PatternFindOptions.Strategy.FIRST)
        .build())
    .build();

Confirmation Pattern

Use CONFIRM strategy to validate matches:

// Find with low threshold, confirm with high threshold
ActionChainOptions confirmChain = new ActionChainOptions.Builder(
    new PatternFindOptions.Builder()
        .setSimilarity(0.7) // Lower threshold
        .build())
    .setStrategy(ActionChainOptions.ChainingStrategy.CONFIRM)
    .then(new PatternFindOptions.Builder()
        .setSimilarity(0.9) // Higher threshold
        .build())
    .build();

Replacing Removed Patterns

ActionChainOptions replaces several composite action patterns that were removed in Brobot 1.1.0:

Instead of MultipleActionsObject (Removed in v1.1.0)

// Old way (NO LONGER AVAILABLE)
// MultipleActionsObject mao = new MultipleActionsObject();
// mao.add(new ActionParameters(clickOptions, buttonCollection));
// mao.add(new ActionParameters(typeOptions, textCollection));

// New way
ActionChainOptions chain = new ActionChainOptions.Builder(
    new ClickOptions.Builder().build())
    .then(new TypeOptions.Builder().build())
    .build();

// Or simpler implicit chaining:
ClickOptions clickThenType = new ClickOptions.Builder()
    .then(new TypeOptions.Builder().build())
    .build();

Instead of ActionResultCombo (Removed in v1.1.0)

// Old way (NO LONGER AVAILABLE)
// ActionResultCombo combo = new ActionResultCombo();
// combo.setActionOptions(clickOptions);
// combo.setResultOptions(findOptions);

// New way
ActionChainOptions chain = new ActionChainOptions.Builder(
    new ClickOptions.Builder().build())
    .then(new PatternFindOptions.Builder().build())
    .build();

No Backward Compatibility

These classes were completely removed with no compatibility layer. Code using MultipleActionsObject, ActionResultCombo, or ActionParameters will not compile in Brobot 1.1.0+.

For migration guidance, see the Migration Guide.

Critical Insight: Object Type Preservation in Chains

Important

When chaining actions that use different object types (e.g., StateImage for find/click vs StateString for type), be aware that the NESTED strategy does NOT preserve all object types through the chain.

The Object Type Problem

The NESTED strategy creates a new ObjectCollection containing only regions from previous matches, discarding other object types like StateStrings. This is a common pitfall when trying to chain find->click->type operations.

// ❌ WRONG: This will fail - strings are lost in NESTED chain
PatternFindOptions findClickType = new PatternFindOptions.Builder()
    .then(new ClickOptions.Builder().build())  // Uses StateImage
    .then(new TypeOptions.Builder().build())   // Needs StateString - but it's lost!
    .build();

ObjectCollection mixed = new ObjectCollection.Builder()
    .withImages(buttonImage)      // Used by find & click
    .withStrings("Hello World")   // Lost during NESTED chaining!
    .build();

// The type action will receive NO strings and fail silently

Solutions for Mixed Object Types

Solution 1: Split Chains by Object Type (RECOMMENDED)

Chain actions that use the same object type, then execute different types separately:

// ✅ CORRECT: Chain find->click (both use StateImage)
PatternFindOptions findClick = new PatternFindOptions.Builder()
    .setPauseAfterEnd(0.5)
    .then(new ClickOptions.Builder()
        .setPauseAfterEnd(0.5)
        .build())
    .build();

// Execute find->click with images
ObjectCollection imageCollection = new ObjectCollection.Builder()
    .withImages(buttonImage)
    .build();
ActionResult clickResult = action.perform(findClick, imageCollection);

// Then execute type separately with strings
TypeOptions typeOptions = new TypeOptions.Builder()
    .setPauseBeforeBegin(0.5)
    .build();
ObjectCollection stringCollection = new ObjectCollection.Builder()
    .withStrings("Hello World")
    .build();
ActionResult typeResult = action.perform(typeOptions, stringCollection);

Solution 2: Use CONFIRM Strategy

The CONFIRM strategy preserves the original ObjectCollection, but may not be semantically appropriate for all workflows:

// ✅ Alternative: CONFIRM strategy preserves all objects
ActionChainOptions chain = new ActionChainOptions.Builder(
    new PatternFindOptions.Builder().build())
    .setStrategy(ActionChainOptions.ChainingStrategy.CONFIRM)  // Preserves ObjectCollection
    .then(new ClickOptions.Builder().build())
    .then(new TypeOptions.Builder().build())
    .build();

// Now mixed collections work
ObjectCollection mixed = new ObjectCollection.Builder()
    .withImages(buttonImage)
    .withStrings("Hello World")
    .build();

Solution 3: Custom Coordination

For complex workflows, manually coordinate between actions:

// ✅ Full control: Coordinate actions manually
ActionResult findResult = action.find(targetImage);
if (findResult.isSuccess()) {
    // Click at the found location
    ObjectCollection clickTarget = new ObjectCollection.Builder()
        .withRegions(findResult.getMatchList().get(0).getRegion())
        .build();
    action.perform(new ClickOptions.Builder().build(), clickTarget);
    
    // Then type
    ObjectCollection typeTarget = new ObjectCollection.Builder()
        .withStrings("Hello World")
        .build();
    action.perform(new TypeOptions.Builder().build(), typeTarget);
}

Best Practices

1. Understand object type flow: Know which actions use which object types
   • Find/Click/Drag: Use StateImage, StateRegion, StateLocation
   • Type: Uses StateString
   • Highlight: Uses any visual object type
2. Chain same-type actions: Group actions that use the same object type
3. Use appropriate strategies:
   • NESTED: For hierarchical searches within same object type
   • CONFIRM: When you need to preserve all object types
4. Set proper delays: Use setPauseAfterEnd() to allow UI updates between actions
5. Handle failures gracefully: Check intermediate results when needed
6. Create reusable patterns: Build utility methods for common chains
7. Keep chains focused: Break complex workflows into smaller, manageable chains

Common Pattern: Find-Click-Type Workflow

This is one of the most common automation patterns - finding an input field, clicking it, and typing text. Here's the definitive guide:

❌ Common Mistake

// This looks logical but FAILS due to object type loss
PatternFindOptions chainedActions = new PatternFindOptions.Builder()
    .then(new ClickOptions.Builder().build())
    .then(new TypeOptions.Builder().build())
    .build();

ObjectCollection everything = new ObjectCollection.Builder()
    .withImages(inputFieldImage)
    .withStrings("user@example.com")
    .build();

// The type action never receives the string!
action.perform(chainedActions, everything);

✅ Correct Implementation

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import io.github.jspinak.brobot.action.Action;
import io.github.jspinak.brobot.action.ActionResult;
import io.github.jspinak.brobot.action.ObjectCollection;
import io.github.jspinak.brobot.action.basic.click.ClickOptions;
import io.github.jspinak.brobot.action.basic.find.PatternFindOptions;
import io.github.jspinak.brobot.action.basic.type.TypeOptions;
import io.github.jspinak.brobot.datatypes.state.stateObject.stateImage.StateImage;

@Component
public class LoginAutomation {
    private static final Logger log = LoggerFactory.getLogger(LoginAutomation.class);

    @Autowired
    private Action action;

    public boolean fillLoginForm(StateImage emailField, StateImage passwordField,
                                 String email, String password) {
        // Step 1: Find and click email field
        PatternFindOptions findClickEmail = new PatternFindOptions.Builder()
            .setSimilarity(0.85)
            .setPauseAfterEnd(0.3)
            .then(new ClickOptions.Builder()
                .setPauseAfterEnd(0.5)  // Wait for field to be active
                .build())
            .build();

        ObjectCollection emailFieldTarget = new ObjectCollection.Builder()
            .withImages(emailField)
            .build();

        ActionResult emailClick = action.perform(findClickEmail, emailFieldTarget);
        if (!emailClick.isSuccess()) {
            log.error("Failed to find/click email field");
            return false;
        }

        // Step 2: Type email
        TypeOptions typeEmail = new TypeOptions.Builder()
            .setPauseBeforeBegin(0.2)
            .setTypeDelay(0.05)  // Human-like typing speed
            .build();

        ObjectCollection emailText = new ObjectCollection.Builder()
            .withStrings(email)
            .build();

        action.perform(typeEmail, emailText);

        // Step 3: Tab to password field (or find-click it)
        action.perform(new TypeOptions.Builder().build(),
            new ObjectCollection.Builder().withStrings("\t").build());

        // Step 4: Type password
        ObjectCollection passwordText = new ObjectCollection.Builder()
            .withStrings(password)
            .build();

        action.perform(typeEmail, passwordText);

        return true;
    }
}

Note: This example is fully compilable with the imports shown above.

Alternative: Using State Transitions

For complex forms, consider using Brobot's state management system instead of chained actions:

@Component
@TransitionSet(state = LoginFormState.class)
public class LoginFormTransitions {

    @Autowired
    private Action action;

    @OutgoingTransition(to = FilledFormState.class)
    public boolean fillLoginForm() {
        // Each action is clear and separated
        // State management handles the flow
        boolean emailFilled = fillEmailField();
        boolean passwordFilled = fillPasswordField();
        boolean submitted = clickSubmit();

        return emailFilled && passwordFilled && submitted;
    }

    private boolean fillEmailField() {
        // Implementation using action.perform()
        return true;
    }

    private boolean fillPasswordField() {
        // Implementation using action.perform()
        return true;
    }

    private boolean clickSubmit() {
        // Implementation using action.perform()
        return true;
    }
}

For more on state transitions, see the States in Brobot guide.

Conditional Action Chaining

For more advanced conditional workflows with proper sequential composition, see the Conditional Action Chains documentation. The ConditionalActionChain class provides:

• The crucial then() method for sequential action composition
• Convenience methods like click(), type(), scrollDown()
• Built-in keyboard shortcuts
• Proper conditional execution logic
• No explicit wait() methods (following model-based principles)

Production Examples

For complete, production-quality examples of action chaining, see:

• library/src/main/java/io/github/jspinak/brobot/action/examples/ActionChainExamples.java

This file contains fully compilable examples including:

• fillLoginForm() - Sequential action patterns
• clickAndVerify() - Click and verify workflow
• findButtonInDialog() - Nested search demonstrations
• findAndConfirm() - Confirmation pattern examples
• complexWorkflow() - Multi-step automation chains
• ChainPatterns class - Reusable chain builders
• buildDynamicChain() - Dynamic chain construction

All examples follow best practices and include complete class context.

Next Steps

Related Guides

• Explore Conditional Action Chains for advanced patterns
• Learn about Complex Workflows for multi-step automation
• Learn about Conditional Actions using RepeatUntilConfig
• See Form Automation for practical examples
• Build Reusable Patterns for your automation library

Reference Documentation

• Check API Reference for complete ActionConfig documentation
• Review Troubleshooting Chains if you encounter issues
• See ActionConfig Overview for conceptual foundation