All Classes and Interfaces
Class
Description
Entry point for executing GUI automation actions in the Brobot model-based framework.
Defines a sequence of actions as data in the Brobot Domain-Specific Language.
Data-driven state transition implementation for the Brobot framework.
Mock durations for different Actions.
Performs the Action and maintenance around the Action such as
- Setting the start time and saving the operation's duration
- Executing pauses before and after the operation
- Determining the success of the operation
- Saving the MatchSnapshots
Any Action that is run independently of this class should
have code to perform maintenance around the Action.
Core interface for all actions in the Brobot model-based GUI automation framework.
Interface for creating different types of logs.
Configuration object that defines how GUI automation actions are executed in Brobot.
Utilities for serializing ActionOptions to and from JSON.
Custom serializer for ActionOptions to handle special cases
An ActionResultsCombos has both an action and an expected result.
Returns the corresponding Action class as specified in ActionOptions.
Determines the text description of an action given the action itself as well as the active states before
and after action execution.
An ActionVector is the translation of the ActionOptions variable into a vector of numbers,
to be used for machine learning.
This translation method uses one hot encoding to encode the type of action performed.
There can be different ways to translate an ActionOption variable to a vector.
Manages a collection of currently active states in the Brobot framework.
Non-image objects are adjusted and added directly to the Matches variable.
Adjust Matches according to the options in ActionOptions.
After a successful click,
Sets the probability that the object remains visible or disappears.
Manages the State repository.
This class is a central location for pixel analysis, for every color-based match strategy.
Defines relative positioning constraints between visual elements in Brobot.
Collection of Anchor objects for defining complex relative positioning constraints.
ANSI escape code constants for colored terminal output in Brobot reporting.
Gets the window of the App in focus.
Validates automation DSL JSON against the automation DSL schema.
Core interface for automation script lifecycle management in the Brobot framework.
Represents the automation configuration for UI buttons in the Brobot Runner.
Utility for Base64 encoding of files and byte arrays in the Brobot framework.
Abstract base class for building GUI automation applications in the Brobot framework.
BasicActions, which run for 1 iteration, require 1 or no Find operations.
Manages Brobot's runtime environment configuration.
Builder for creating custom BrobotEnvironment configurations.
Mixin for Brobot Image class to prevent serializing BufferedImage.
Global configuration settings for the Brobot model-based GUI automation framework.
This class implements the SmartLifecycle interface to perform actions after
the Spring context is initialized.
Mixin for BufferedImage to prevent serializing internal classes.
Comprehensive BufferedImage operations for the Brobot GUI automation framework.
Represents a method call in the Brobot Domain-Specific Language (DSL).
Main entry point for business rule validation.
Represents a UI button configuration for the desktop runner that calls automation functions.
Represents position information for a button in the UI.
Represents styling information for a button.
Captures the screenshot from the screen where the mouse is located and saves it to the given directory.
Saves a series of screenshots.
DynamicImages classify as probabilities, standard Images have 100% pixel probability when found.
Performs mouse click operations on GUI elements in the Brobot framework.
Wrapper class for Click, handles real and mock clicks.
Converts ClickType to a Sikuli Button.
ClickUntil clicks Matches until a condition occurs or the operation times out.
Manages color profiles across multiple color spaces for pattern matching in Brobot.
ColorInfo has the min, max, mean, and stddev for a single ColorValue, such as Hue, Saturation, or Value.
Mixin for ColorModel to prevent serializing internal classes.
A ColorSchema is a representation of a color cluster.
ColorStatProfile is a color in a specific color space (HSV, BGR, etc.).
Provides factory methods for commonly used ActionOptions configurations.
It is useful to create one or more CommonActions classes for your applications
in order to reduce code redundancy and make your code more readable.
Currently ActionResultCombos are set up for click operations as the first action.
This class defines some typical configurations for Drag and
allows for multiple drag operations.
This is an example of a more complex custom action.
This class should make it simpler to create common StateTransitions.
CompositeActions are built from BasicActions.
Exception thrown when there is an error in processing Brobot Runner configurations.
Exception thrown when configuration validation fails.
Main validator class for Brobot configurations.
Performs nested Find actions, in the order they appear in the FindActions list in
ActionOptions.
Implements continuous state-based automation that monitors and responds to GUI changes.
This class creates a deep copy of most of the fields in ActionOptions.
Mixin for DataBuffer to prevent serializing internal classes.
Each region has 4 borders that can be defined: top, bottom, left, right.
Helper functions for various Define classes
Defines a Region as the largest rectangle including all Matches and Locations.
Defines a Region as the smallest rectangle produced by Matches and Locations.
Defines a Region as the largest rectangle produced by Matches and Locations.
Defines a Region with the Define method specified in the ActionOptions.
Defines a Region around a Match.
Defines a Region using the active Window as a reference.
Represents directional relationships in state transitions and navigation.
Completes a whole Transition (FromTransition and ToTransition),
including all States to activate and exit.
The DoUntilActionObject contains both ActionOptions and ObjectCollections for both
'do' and 'until' Actions.
Performs drag-and-drop operations between GUI elements in the Brobot model-based automation framework.
Wrapper class for Drag, handles real and mock drags.
Deprecated.
As of version 1.0.7, use Pattern objects with color analysis instead
IllustrationScenes store their filenames before they are written to file.
Core pattern matching action that locates GUI elements on the screen.
Finds all Matches for all Patterns in the Brobot Image.
The steps for color classification are:
1.
Retrieve a Find function with its Find type.
Searches a scene for pattern and word matches.
TimeBetweenIndividualActions is the option in ActionOptions that gives the time (in seconds)
between the first, second, and third screenshots.
Sends the Image object to either FindImage or FindRIP depending on whether the Image location can vary.
Finds the regions with dynamic pixels.
The image to compare is in ObjectCollections #1 (0), in the list of StateImage objects.
Validates references related to automation functions, including function calls,
parameters, and references to functions from other components.
Validates business rules related to automation functions.
Returns the 'from' and 'to' Locations to be used in Drag.
JavaCV-based image acquisition and conversion utilities for Brobot.
This class is primarily for color analysis.
Scenes are used for
- finding color
- image segmentation (classification)
- action illustration
In Brobot version 1.0.7, scenes are used for all Find operations.
Wrapper class for GetText, handles real and mock text queries.
Divides a screen region into a matrix of cells for systematic interaction.
Deprecated.
Use
BrobotEnvironment instead.Highlights visual matches on screen in the Brobot model-based GUI automation framework.
Wrapper class for Highlight, performs real or mock highlights.
Has the mask for one part of each image, i.e.
Divides a region into five subregions to facilitate histogram and color Find operations.
Container for visual debugging and analysis illustrations in Brobot.
Physical representation of an image in the Brobot GUI automation framework.
Custom deserializer for Image class that creates an Image with an empty BufferedImage.
Validates that image resources referenced in the project configuration actually exist
and are valid image files.
Custom serializer for Image class that omits the BufferedImage.
Initialization service for the Brobot model-based GUI automation framework.
Given sets of possible active States, this class searches for these States
to set active States.
Core interface for state transitions in the Brobot model-based GUI automation framework.
Code-based state transition implementation for the Brobot framework.
Configuration for JSON processing in the Brobot application.
JsonParser provides methods for working with JSON in a robust way.
Utility class for working with JSON paths and extracting data from complex JSON structures.
Utility class that provides additional JSON functionality focused on
handling problematic Java classes (circular references, module system issues).
Presses and holds keyboard keys in the Brobot model-based GUI automation framework.
KeyDown in SikuliX (Java Robot) does not actually hold the key down.
Releases previously pressed keyboard keys in the Brobot model-based GUI automation framework.
Wrapper class for KeyUp, works for real or mocked operations.
KmeansCluster represents a cluster of points in a k-means analysis corresponding to one of the centers.
KmeansProfile represents a k-means analysis for one number of means and one color schema (BGR, HSV, etc.).
Creates a series of kmeans profiles for commonly used number of centers (2-10)
Calculating this up-front will save time and CPU/GPU.
Represents a point on the screen in the Brobot model-based GUI automation framework.
Comprehensive utility methods for Location manipulation and calculations in Brobot.
An interface for components that wish to persist log data produced by the library.
Flexible builder for composing multiple OpenCV Mat objects into a single image.
Represents a successful pattern match found on the screen in the Brobot model-based GUI automation framework.
Comprehensive results container for all action executions in the Brobot framework.
Manages the Matches throughout the lifecycle of the action.
Utilities for serializing Matches to and from JSON.
Custom serializer for Matches to handle special cases
Match objects do not retain data on Pattern, StateObject, etc.
Maintains historical match data for pattern learning and mock execution in the Brobot framework.
Creates Match objects when probability is being used instead of Snapshots.
Mixin for Match class to prevent serializing fields that could cause null reference issues.
Records match results and context at a specific point in time for the Brobot framework.
Comprehensive utility class for OpenCV Mat operations in Brobot.
3d versions of common OpenCV operations that only accept one channel.
Custom serializer for Mat objects
Mainly for testing, this class provides code to easily visualize Mats.
Returns snapshots with matching actions and states.
Takes care of setting probabilities for States and State objects
Tracks mock operation counts for controlled testing scenarios.
Mock text for GetText Actions using Snapshots.
Provides simulated time control for mock testing environments in Brobot.
Presses and holds a mouse button in the Brobot model-based GUI automation framework.
Wrapper class for MouseDown, works for real or mock actions.
Releases a mouse button in the Brobot model-based GUI automation framework.
Wrapper class for MouseUp, works for real or mocked actions.
Wrapper class for MouseWheel, works for real or mocked actions.
Moves the mouse to one or more locations in the Brobot model-based GUI automation framework.
Wrapper class for MoveMouse, works for real or mocked actions.
Helper function for Composite Actions
NOT WORKING! Sikuli appears not to allow mouse movement after a mouse down operation.
This is an example of how to create an Action using the MultipleActions class.
Performs nested Find actions, in the order they appear in the FindActions list in
ActionOptions.
Represents the absence of a match in the Brobot model-based GUI automation framework.
A default, do-nothing implementation of LogSink.
Special state for handling stateless objects in the Brobot framework.
Container for GUI elements that serve as targets for automation actions in Brobot.
Utilities for serializing ObjectCollection to and from JSON.
Custom serializer for ObjectCollection to handle special cases
Creates two offset grids that overlap to provide comprehensive coverage of grid-like interfaces.
Generic immutable container for holding two related values as a single unit.
ParameterCollections help Brobot calibrate specific parameters with respect
to success conditions.
Represents a navigation path between states in the Brobot model-based GUI automation framework.
Implements graph traversal algorithms to find navigation paths between States.
Manages path scoring and recovery after failed state traversals.
Collection of navigation paths in the Brobot model-based GUI automation framework.
Represents a visual template for pattern matching in the Brobot GUI automation framework.
Controls execution permissions and modes in the Brobot framework.
Holds the results of all per-pixel color analysis for a {scene, ColorProfile} pair.
Contains a series of PixelAnalysis objects that comprise all analysis of a {scene, StateImage} pair.
Represents a relative position within a rectangular area using percentage coordinates.
Defines standard relative positions within a rectangular area.
Comprehensive container for an entire Brobot automation project.
Represents the project configuration settings and references to automation functions.
Validates project configuration JSON against the project schema.
It divides a region into
grid cells, some overlapping, and counts the points that are within the cells.
This class is meant mostly for testing state structures and state management,
although it can also be used for bootstrapping simple projects.
Mixin for Raster to prevent serializing internal classes.
Mixin for java.awt.geom.Rectangle2D to prevent possible circular references.
Custom serializer for JavaCV Rect objects to prevent circular references.
Main facade for cross-reference validation between configuration entities.
Represents a rectangular area on the screen in the Brobot model-based GUI automation framework.
Mixin for Region class to prevent circular references.
Comprehensive utility methods for Region manipulation and analysis in Brobot.
Centralized reporting system with configurable output levels for the Brobot framework.
Returns a List with the first element the Action Matches and the second element the Results Matches
ActionResultsCombos are typically:
- an Action such as CLICK
- a Result from an operation such as FIND
The parameters and results only get recorded if the Action is successful.
Represents a captured screenshot or screen state as a searchable pattern.
A collection of PixelAnalysisCollection objects, each of which contains all analysis of a {scene, StateImage} pair.
Records all scenes associated with an image, as found in SceneCombinations.
Manages JSON schemas for the Brobot Runner configuration system.
Entry point for schema validation operations.
Scrolls the mouse wheel in the Brobot model-based GUI automation framework.
Manages multiple search areas for pattern matching in the Brobot framework.
Mixin for SearchRegions to prevent circular references.
Utility methods for working with SearchRegions objects.
Represents an angular sector (arc) in a circular coordinate system.
Select is a custom Action built using the custom SelectActionObject.
Set the hidden States after a successful State transition.
Mats used to illustrate the image and its color profiles can be stored together with
the image and referenced for illustration when needed.
Mixin for SikuliLocation class to prevent circular references.
Mixin for SikuliScreen class to prevent circular references.
Defines special state types with reserved IDs in the Brobot framework.
Represents a distinct configuration of the GUI in the Brobot model-based automation framework.
Marker interface for state enumerations in the Brobot framework.
Discovers active states through visual pattern matching in the Brobot framework.
Interface for implementing custom state handling logic in automation scripts.
States can be illustrated in various ways.
Visual pattern identifier for States in the Brobot model-based GUI automation framework.
Represents a log entry for a state image.
Represents a meaningful screen coordinate associated with a specific state in Brobot.
Service responsible for managing state-related operations in the Brobot framework.
Maintains the runtime memory of active States in the Brobot framework.
Core interface for all objects that belong to states in the Brobot framework.
Lightweight reference to StateObject instances in the Brobot framework.
Validates references related to states, including state elements, transitions,
and references to states from other components.
Represents a meaningful screen area associated with a specific state in Brobot.
Represents text input associated with a specific state in the Brobot framework.
Represents persistent text that uniquely identifies a state in GUI automation.
Container for all transitions associated with a specific State in the Brobot framework.
Finds the correct Transition from one State to another,
taking into account the hidden States.
Maintains a comprehensive graph of state transitions for efficient navigation queries.
We want to go to a specific State, the target State.
Manages the StateTransitions repository and retrieves
StateTransitions given a state name.
Calculates the similarity of 2 Strings.
Defines the success condition for individual Actions.
The TestManager is in charge of the progress of an automation application.
Tests can be used for mocks or for live execution.
Brobot's AUT testing strategy is to go through all states looking for missing
images or faulty transitions.
This interface manages the logging session.
Represents text extracted from GUI elements with inherent OCR variability.
The TextSelector returns text from the stochastic Text variable (Strings are stored in a List),
which it believes has the highest probability of being the actual text on the screen.
This class returns an object containing the StateTransitions objects and StateTransition objects that
we are interested in for a specific transition from one State to another.
Validates business rules related to state transitions.
Moves through the Paths to reach the target State.
Aims to visualize the State hierarchy.
Types text to the window in focus in the Brobot model-based GUI automation framework.
Wrapper class for TypeText, works with real or mocked actions.
Represents the initial uncertain state in Brobot's state management system.
Use the defined regions (if they exist) of objects as MatchObjects
Represents an error found during configuration validation.
Contains the results of a validation operation.
Represents the severity of a validation error.
Waits for visual elements to disappear from the screen in the Brobot model-based GUI automation framework.
Mixin for WritableRaster to prevent serializing internal classes.