Where do I go from here?

Now that you have a state structure, you can start writing the code for your automation program. How you do this is ultimately up to you, but I'll share here how I write my Brobot code.

Activities classes

I usually create a class with the name someClass + "Activities" for actions that happen within a specific class. We have a class called BlackSpriritsAdventure inside the folder blackSpiritsAdventure. If I want to click on the Acquire button, I would first create a class called BlackSpriritsAdventureActivities within the same folder.

This class would inject 3 dependencies in its constructor: BlackSpriritsAdventure, StateTransitionsManagement, and Action. If you have correctly defined the transitions in each of your transitions classes, you can use the StateTransitionsManagement class to open any state in your application. It doesn't matter where you are currently in the environment, as long as you have defined the transitions correctly, you can open any state.

The BlackSpriritsAdventure dependency is necessary only to reference the objects in it (here, the Acquire button). To open the state with the StateTransitionsManagement dependency, you don't need to inject the BlackSpriritsAdventure state class. This is because the StateTransitionsManagement class's open method takes an Enum object as a parameter. The Enum object is the name of the state you want to open (here, BLACKSPIRITSADVENTURE), and it is defined as a public variable. If we just wanted to open the BlackSpriritsAdventure state, we wouldn't need a BlackSpriritsAdventureActivities class; we could open it with one line of code:

stateTransitionsManagement.openState(BLACKSPIRITSADVENTURE);

Action is used to perform actions on the screen.

Our class BlackSpriritsAdventureActivities would look like this:

@Component
public class BlackSpriritsAdventureActivities {

    private BlackSpriritsAdventure blackSpriritsAdventure;
    private StateTransitionsManagement stateTransitionsManagement;
    private Action action;

    public BlackSpriritsAdventureActivities(BlackSpriritsAdventure blackSpriritsAdventure,
                                            StateTransitionsManagement stateTransitionsManagement,
                                            Action action) {
        this.blackSpriritsAdventure = blackSpriritsAdventure;
        this.stateTransitionsManagement = stateTransitionsManagement;
        this.action = action;
    }

    public boolean acquire() {
        if (!stateTransitionsManagement.openState(BLACKSPIRITSADVENTURE)) return false;
        return action.perform(CLICK, blackSpriritsAdventure.getAcquireButton());
    }
}

Action basics

The Action class is used to perform actions on the screen. It has a method called perform that takes an ActionOptions object and an ObjectCollection object. The ActionOptions object defines the action to be performed, and the ObjectCollection object defines the objects on the screen that the action will be performed on. The ActionOptions object has a builder that allows you to define the action to be performed. The ObjectCollection object has a builder that allows you to define the objects on the screen that the action will be performed on. The ObjectCollection object can be created with a single object, or with a list of objects.

Action tips

When using a standard action; for example, a single left click without any modifiers, you can use the following code to perform the action:

action.perform(CLICK, objectCollection);

If you wish to use a non-standard action, you will need to create an ActionOptions variable and specific its parameters. Then you would pass it to the perform method along with the ObjectCollection object.

ActionOptions actionOptions = ActionOptions.builder()
        .actionType(CLICK)
        .clickType(RIGHT)
        .build();
action.perform(actionOptions, objectCollection);

The same applies to the ObjectCollection object. If you wish to use a single image, you can just pass the image to the perform method as such:

action.perform(actionOptions, image);

If you want the action to be performed on multiple objects, you can use the ObjectCollection builder to create a list of objects.

ObjectCollection objectCollection = ObjectCollection.builder()
        .withImages(image1, image2, image3)
        .withRegions(region1, region2, region3)
        .build();
action.perform(actionOptions, objectCollection);

To perform a standard click on one image:

action.perform(CLICK, image);