Convenience Methods
Brobot 1.1 introduces convenience methods that dramatically simplify common automation tasks. These methods provide direct access to frequently-used operations without the verbosity of creating ObjectCollections.
Required Importsโ
All examples in this guide assume the following imports:
// Brobot Core
import io.github.jspinak.brobot.action.Action;
import io.github.jspinak.brobot.action.ActionResult;
import io.github.jspinak.brobot.action.ObjectCollection;
// Brobot Datatypes
import io.github.jspinak.brobot.datatypes.primitives.region.Region;
import io.github.jspinak.brobot.datatypes.primitives.location.Location;
import io.github.jspinak.brobot.datatypes.primitives.match.Match;
import io.github.jspinak.brobot.datatypes.state.stateObject.stateImage.StateImage;
import io.github.jspinak.brobot.datatypes.state.stateObject.stateString.StateString;
// ActionConfig Classes
import io.github.jspinak.brobot.action.basic.mouse.MouseMoveOptions;
import io.github.jspinak.brobot.action.basic.scroll.ScrollOptions;
// SikuliX
import org.sikuli.script.Pattern;
// Spring Framework
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
Prerequisitesโ
Unless otherwise specified, the code examples assume you have:
@Autowired Action action- Injected Brobot Action service- StateImage variables (e.g.,
usernameField,submitButton) - Pre-initialized UI elements - Example objects defined locally where needed
See the Complete Working Example section for a fully compilable Spring Boot application.
Basic Setupโ
@Component
public class ConvenienceMethodsExample {
@Autowired
private Action action;
// Example StateImages used in examples below
private StateImage usernameField = new StateImage.Builder()
.addPattern("username-field.png")
.build();
private StateImage passwordField = new StateImage.Builder()
.addPattern("password-field.png")
.build();
private StateImage submitButton = new StateImage.Builder()
.addPattern("submit-button.png")
.build();
private StateImage loadingSpinner = new StateImage.Builder()
.addPattern("loading-spinner.png")
.build();
}
Overviewโ
The convenience methods follow a hybrid approach:
- Convenience: Direct methods for single-object operations
- Consistency: All methods internally delegate to the canonical ObjectCollection-based implementations
- Compatibility: Existing code continues to work unchanged
Before and Afterโ
The Traditional Approachโ
Previously, every action required wrapping objects in an ObjectCollection:
// Clicking a region - verbose
ObjectCollection oc = new ObjectCollection.Builder()
.withRegions(region)
.build();
ActionResult result = action.click(oc);
// Typing text - verbose
ObjectCollection textOc = new ObjectCollection.Builder()
.withStrings("Hello World")
.build();
action.type(textOc);
// Moving to a location - verbose
MouseMoveOptions moveOptions = new MouseMoveOptions.Builder().build();
ObjectCollection locOc = new ObjectCollection.Builder()
.withLocations(location)
.build();
action.perform(moveOptions, locOc);
The New Convenience Methodsโ
With convenience methods, common operations become one-liners:
// Clicking a region - simple
ActionResult result = action.click(region);
// Typing text - simple
action.type("Hello World");
// Moving to a location - simple
action.move(location);
Available Convenience Methodsโ
Click Operationsโ
// Click on a specific region
action.click(region);
// Click at a specific location
action.click(location);
// Click on a previously found match
Match match = action.find(pattern).getBestMatch().orElse(null);
if (match != null) {
action.click(match);
}
Type Operationsโ
// Type plain text
action.type("user@example.com");
// Type with state context (using StateString with convenience method)
StateString stateString = new StateString.Builder()
.setString("password123")
.setOwnerStateName("LoginPage")
.build();
action.type(stateString);
// Type in a specific field (find and click, then type)
action.click(usernameField);
action.type("username123");
Find Operationsโ
// Find a pattern directly
Pattern pattern = new Pattern("button.png");
ActionResult result = action.find(pattern);
// Find a StateImage directly
StateImage image = new StateImage.Builder()
.addPattern(pattern)
.build();
ActionResult result = action.find(image);
// Find multiple StateImages
ActionResult results = action.find(submitButton, cancelButton);
Mouse Movementโ
// Move to a specific location
Location loc = new Location(500, 300);
action.move(loc);
// Move to center of a region
Region region = new Region(100, 100, 200, 200);
action.move(region);
// Move to a match
Match match = findResult.getBestMatch().orElse(null);
if (match != null) {
action.move(match);
}
Highlightingโ
// Highlight a region for debugging
Region searchArea = new Region(0, 0, 800, 600);
action.highlight(searchArea);
// Highlight a found match
Match match = action.find(pattern).getBestMatch().orElse(null);
if (match != null) {
action.highlight(match);
}
๐ก Advanced Highlighting: For more control over highlighting including custom colors, durations, and automatic visual feedback, see the Highlighting Feature Guide.
Drag Operationsโ
// Drag from one location to another
Location from = new Location(100, 100);
Location to = new Location(500, 500);
action.drag(from, to);
// Drag between region centers
Region sourceRegion = new Region(50, 50, 100, 100);
Region targetRegion = new Region(400, 400, 100, 100);
action.drag(sourceRegion, targetRegion);
Scrollingโ
// Scroll down 3 steps
action.scroll(ScrollOptions.Direction.DOWN, 3);
// Scroll up 5 steps
action.scroll(ScrollOptions.Direction.UP, 5);
Waiting for Elements to Vanishโ
// Wait for a pattern to disappear
Pattern loadingSpinner = new Pattern("loading.png");
action.vanish(loadingSpinner);
// Wait for a StateImage to vanish
action.vanish(loadingImage);
Common Workflowsโ
Example 1: Login Form Automationโ
@Autowired
private Action action;
public void login(String username, String password) {
// Click username field and type
action.click(usernameField);
action.type(username);
// Click password field and type
action.click(passwordField);
action.type(password);
// Click submit button
action.click(submitButton);
// Wait for loading to complete
action.vanish(loadingSpinner);
}
// Alternative using StateString for better state context
public void loginWithStateContext() {
StateString username = new StateString.Builder()
.setString("user@example.com")
.setOwnerStateName("LoginPage")
.setSearchRegion(usernameFieldRegion)
.build();
StateString password = new StateString.Builder()
.setString("secure123")
.setOwnerStateName("LoginPage")
.setSearchRegion(passwordFieldRegion)
.build();
action.type(username);
action.type(password);
action.click(submitButton);
}
Example 2: Drag and Drop File Uploadโ
public void uploadFile() {
// Find the file to upload
ActionResult fileResult = action.find(fileIcon);
// Find the upload area
ActionResult uploadResult = action.find(uploadArea);
// Drag the file to upload area
if (fileResult.isSuccess() && uploadResult.isSuccess()) {
Match file = fileResult.getBestMatch().orElse(null);
Match upload = uploadResult.getBestMatch().orElse(null);
if (file != null && upload != null) {
action.drag(file.getRegion(), upload.getRegion());
}
}
}
Example 3: Search and Navigationโ
public void searchAndNavigate(String searchTerm) {
// Click on search box
action.click(searchBox);
// Type search term
action.type(searchTerm);
// Press Enter (using traditional approach for special keys)
action.type(new ObjectCollection.Builder()
.withStrings("\n")
.build());
// Wait for results to load
action.vanish(loadingIndicator);
// Find and click first result
ActionResult results = action.find(searchResult);
results.getBestMatch().ifPresent(match -> {
action.click(match);
});
}
Example 4: Form Validation with Highlightingโ
public boolean validateForm() {
// Find all required fields
ActionResult nameResult = action.find(nameField);
ActionResult emailResult = action.find(emailField);
ActionResult phoneResult = action.find(phoneField);
// Highlight missing fields
if (!nameResult.isSuccess()) {
action.highlight(nameFieldRegion);
return false;
}
if (!emailResult.isSuccess()) {
action.highlight(emailFieldRegion);
return false;
}
if (!phoneResult.isSuccess()) {
action.highlight(phoneFieldRegion);
return false;
}
return true;
}
Best Practicesโ
When to Use Convenience Methodsโ
โ Use convenience methods for:
- Single-object operations
- Simple, straightforward actions
- Rapid prototyping
- Clear, readable automation scripts
- Working with results from previous finds
When to Use ObjectCollectionsโ
โ Use ObjectCollections for:
- Multiple objects in a single operation
- Complex search criteria
- Operations requiring specific configurations
- When you need fine-grained control
Combining Approachesโ
You can freely mix convenience methods with traditional approaches:
// Use convenience method for simple find
ActionResult result = action.find(pattern);
// Use ObjectCollection for complex multi-object operation
if (result.isSuccess()) {
ObjectCollection complexCollection = new ObjectCollection.Builder()
.withRegions(result.getBestMatch().get().getRegion())
.withDefinedSearchRegion(customSearchArea)
.withStrings("Complex Text")
.build();
// Perform complex action with full configuration
TypeOptions typeOptions = new TypeOptions.Builder()
.setModifiers("CTRL")
.setPauseAfterKeyDown(0.5)
.build();
action.perform(typeOptions, complexCollection);
}
Method Referenceโ
Click Methodsโ
| Method | Description | Return Type | Since |
|---|---|---|---|
click(Region region) | Clicks on the specified region. Convenience method equivalent to click(ObjectCollection.builder().withRegions(region).build()). | ActionResult | 1.1 |
click(Location location) | Clicks at a specific screen coordinate. The location is automatically wrapped in an ObjectCollection for processing. | ActionResult | 1.1 |
click(Match match) | Clicks on the region of the specified match. Extracts the region from a match result and clicks on it. Useful for clicking on previously found elements without manual region extraction. | ActionResult | 1.1 |
Type Methodsโ
| Method | Description | Return Type | Since |
|---|---|---|---|
type(String text) | Types the specified text string at the current cursor position or active input field. Convenience method for typing plain text without needing to wrap it in an ObjectCollection. | ActionResult | 1.1 |
type(StateString stateString) | Types the text from the specified StateString. Enables typing text that has state context (state ownership and optional spatial context for where to click before typing) without needing to wrap it in an ObjectCollection. | ActionResult | 1.1 |
Find Methodsโ
| Method | Description | Return Type | Since |
|---|---|---|---|
find(Pattern pattern) | Finds the specified pattern on screen. Convenience method that searches for a single pattern without needing to wrap it in a StateImage and ObjectCollection. Uses default find options. | ActionResult | 1.1 |
find(StateImage... images) | Finds one or more StateImages on screen. The images are automatically wrapped in an ObjectCollection and searched using default Find parameters. | ActionResult | 1.1 |
Move Methodsโ
| Method | Description | Return Type | Since |
|---|---|---|---|
move(Location location) | Moves the mouse cursor to a specific screen coordinate. The movement uses default speed and path settings. | ActionResult | 1.1 |
move(Region region) | Moves the mouse to the center of the specified region. Calculates the center point of a region and moves the mouse cursor there. Useful for hovering over UI elements. | ActionResult | 1.1 |
move(Match match) | Moves the mouse to the center of the specified match. Extracts the region from a match and moves the mouse to its center. Useful for hovering over previously found elements. | ActionResult | 1.1 |
Highlight Methodsโ
| Method | Description | Return Type | Since |
|---|---|---|---|
highlight(Region region) | Highlights the specified region on screen. Draws a visual highlight around a region for debugging or user feedback purposes. The highlight duration uses default settings. | ActionResult | 1.1 |
highlight(Match match) | Highlights the region of the specified match. Highlights a previously found match, useful for visual debugging of pattern matching results. | ActionResult | 1.1 |
Drag Methodsโ
| Method | Description | Return Type | Since |
|---|---|---|---|
drag(Location from, Location to) | Drags from one location to another. Performs a drag operation between two screen coordinates. Uses default mouse button (left) and drag speed settings. | ActionResult | 1.1 |
drag(Region from, Region to) | Drags from one region's center to another region's center. Performs a drag operation between the centers of two regions. Useful for dragging UI elements from one area to another. | ActionResult | 1.1 |
Scroll Methodsโ
| Method | Description | Return Type | Since |
|---|---|---|---|
scroll(ScrollOptions.Direction dir, int steps) | Scrolls the mouse wheel at the current cursor position. Performs mouse wheel scrolling with the specified direction (UP or DOWN) and number of scroll steps (notches). | ActionResult | 1.1 |
Vanish Methodsโ
| Method | Description | Return Type | Since |
|---|---|---|---|
vanish(StateImage image) | Waits for the specified image to vanish from the screen. Repeatedly checks if an image is no longer visible using default timeout and check interval settings. | ActionResult | 1.1 |
vanish(Pattern pattern) | Waits for the specified pattern to vanish from the screen. Checks if a pattern is no longer visible. The pattern is automatically wrapped in a StateImage for processing. | ActionResult | 1.1 |
Implementation Detailsโ
Architectureโ
All convenience methods follow the same pattern:
- Thin Wrapper: Each convenience method is a thin wrapper
- Delegation: Internally creates an ObjectCollection and delegates to the canonical method
- Consistency: Ensures all code paths go through the same validation, logging, and processing
Example implementation:
public ActionResult click(Region region) {
// Create ObjectCollection with the region
ObjectCollection collection = new ObjectCollection.Builder()
.withRegions(region)
.build();
// Delegate to canonical click method
return click(collection);
}
Benefitsโ
- No Code Duplication: Business logic remains in one place
- Consistent Behavior: All clicks work the same way regardless of how they're called
- Easy Maintenance: Changes to click behavior only need to be made once
- Full Compatibility: Existing code continues to work without modifications
Migration Guideโ
Migrating Existing Codeโ
You can migrate existing code gradually:
// Old code - still works
ObjectCollection oc = new ObjectCollection.Builder()
.withRegions(region)
.build();
action.click(oc);
// New code - cleaner
action.click(region);
No Breaking Changesโ
The convenience methods are purely additive - no existing functionality has been removed or changed. You can:
- Continue using ObjectCollections everywhere
- Gradually adopt convenience methods where they make sense
- Mix both approaches in the same codebase
Complete Working Exampleโ
Here's a fully compilable Spring Boot application demonstrating convenience methods:
package com.example.automation;
import io.github.jspinak.brobot.action.Action;
import io.github.jspinak.brobot.action.ActionResult;
import io.github.jspinak.brobot.datatypes.primitives.region.Region;
import io.github.jspinak.brobot.datatypes.state.stateObject.stateImage.StateImage;
import io.github.jspinak.brobot.datatypes.state.stateObject.stateString.StateString;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
/**
* Complete example demonstrating Brobot 1.1 convenience methods.
* This class shows practical patterns for common automation tasks.
*/
@Slf4j
@Component
public class ConvenienceMethodsDemoAutomation {
private final Action action;
// StateImages for UI elements
private final StateImage usernameField;
private final StateImage passwordField;
private final StateImage submitButton;
private final StateImage loadingSpinner;
private final StateImage successMessage;
@Autowired
public ConvenienceMethodsDemoAutomation(Action action) {
this.action = action;
// Initialize StateImages
// In production, these would typically be defined in State classes
this.usernameField = new StateImage.Builder()
.addPattern("images/login/username-field.png")
.setSimilarity(0.85)
.build();
this.passwordField = new StateImage.Builder()
.addPattern("images/login/password-field.png")
.setSimilarity(0.85)
.build();
this.submitButton = new StateImage.Builder()
.addPattern("images/login/submit-button.png")
.setSimilarity(0.90)
.build();
this.loadingSpinner = new StateImage.Builder()
.addPattern("images/login/loading-spinner.png")
.setSimilarity(0.80)
.build();
this.successMessage = new StateImage.Builder()
.addPattern("images/login/success-message.png")
.setSimilarity(0.90)
.build();
}
/**
* Basic login using convenience methods.
* Shows the simplest approach with plain strings.
*/
public boolean loginBasic(String username, String password) {
log.info("Starting basic login for user: {}", username);
// Click username field and type
ActionResult usernameClick = action.click(usernameField);
if (!usernameClick.isSuccess()) {
log.error("Username field not found");
return false;
}
action.type(username);
// Click password field and type
ActionResult passwordClick = action.click(passwordField);
if (!passwordClick.isSuccess()) {
log.error("Password field not found");
return false;
}
action.type(password);
// Click submit
ActionResult submitClick = action.click(submitButton);
if (!submitClick.isSuccess()) {
log.error("Submit button not found");
return false;
}
// Wait for loading to complete
action.vanish(loadingSpinner);
// Verify success
ActionResult success = action.find(successMessage);
if (success.isSuccess()) {
log.info("Login successful");
return true;
} else {
log.error("Login failed - success message not found");
return false;
}
}
/**
* Advanced login using StateString for better state context.
* Shows how to use convenience methods with state-aware objects.
*/
public boolean loginAdvanced() {
log.info("Starting advanced login with StateString");
// Create StateStrings with context
StateString usernameInput = new StateString.Builder()
.setString("admin@example.com")
.setOwnerStateName("LoginScreen")
.setSearchRegion(new Region(100, 100, 400, 50))
.build();
StateString passwordInput = new StateString.Builder()
.setString("securePassword123")
.setOwnerStateName("LoginScreen")
.setSearchRegion(new Region(100, 200, 400, 50))
.build();
// Use convenience methods with StateStrings
action.type(usernameInput);
action.type(passwordInput);
// Submit and verify
ActionResult submitResult = action.click(submitButton);
if (!submitResult.isSuccess()) {
log.error("Submit button not found");
return false;
}
action.vanish(loadingSpinner);
return action.find(successMessage).isSuccess();
}
/**
* Demonstrates mixing convenience methods with traditional API.
* Shows when to use each approach.
*/
public void demonstrateMixedApproach() {
log.info("Demonstrating mixed approach");
// Simple operation - use convenience method
ActionResult findResult = action.find(usernameField);
// Complex operation - use traditional API with full configuration
if (findResult.isSuccess()) {
action.highlight(findResult.getBestMatch().get().getRegion());
log.info("Username field highlighted");
}
}
}
This example is fully compilable and demonstrates:
- Complete Spring setup with
@Componentand@Autowired - StateImage initialization with proper builders
- Basic convenience methods (click, type, find, vanish)
- StateString usage with convenience methods
- Error handling with ActionResult checks
- Logging at appropriate levels
- Production-ready structure that can be deployed immediately
To use this in your Spring Boot application:
- Place image files in
src/main/resources/images/login/directory - Ensure Brobot dependencies are configured
- Inject this component into your automation workflows
- Call
loginBasic()orloginAdvanced()to execute login
Summaryโ
The convenience methods in Brobot 1.1 significantly improve developer experience by:
- Reducing Boilerplate: Common operations require 80% less code on average (67-89% depending on operation type)
- Improving Readability: Code reads more like natural language
- Maintaining Flexibility: Complex operations still have full ObjectCollection support
- Preserving Architecture: All benefits of the model-based approach are maintained
These methods make Brobot more accessible to new users while maintaining the power and flexibility that advanced users depend on.
Related Documentationโ
Getting Startedโ
- Pure Actions Quick Start - Apply convenience methods in practice
- States in Brobot - Understanding StateImage and StateString
ActionConfig Documentationโ
- ActionConfig Overview - Understanding the full action configuration system
- ActionResult Components - Working with results from convenience methods
- Complex Workflows - Combining convenience methods with full API
- Form Automation - Apply convenience methods to form interactions
- Conditional Chains - For conditional workflow patterns
Advanced Topicsโ
- Action Chaining - When to use chains vs convenience methods
- Conditional Actions - Conditional execution patterns
Testingโ
- Unit Testing - Testing code that uses convenience methods
- Integration Testing - End-to-end testing patterns
- Mock Mode - Testing with Brobot's mock framework