Package io.github.jspinak.brobot.action.internal.find.match

Class MatchContentExtractor

java.lang.Object
io.github.jspinak.brobot.action.internal.find.match.MatchContentExtractor
@Component public class MatchContentExtractor extends Object
Captures visual and text content from matched regions after find operations.

This component is responsible for post-processing matches to extract their visual representations (Mat/BufferedImage) and text content from the scenes where they were found. It operates after all match adjustments (fusion, position shifts) have been completed to ensure accurate content capture from the final match regions.

Key features:

  • Captures images from specific scenes based on action options
  • Extracts text using OCR when required
  • Handles scene selection for multi-scene captures
  • Respects text extraction conditions based on action type

The component ensures that matches contain the actual visual content they represent, enabling subsequent operations that require image data or text extraction from the matched regions.

See Also:

  • Constructor Details

    • MatchContentExtractor

      public MatchContentExtractor(ExecutionModeController mockOrLive)
      Creates a new MatchContentExtractor instance.
      Parameters:
      mockOrLive - Service for executing operations in mock or live mode

  • Method Details

    • set

      public void set(ActionResult matches)
      Captures both visual and text content from matched regions.

      This method should run after all adjustments are made to the match objects, including position shifts, match fusion, and other transformations specified by ActionConfig. The order is critical because these adjustments affect the underlying regions from which content is captured.

      The method performs two operations:

      1. Captures the visual content (Mat/BufferedImage) from the scene
      2. Extracts text using OCR (currently commented out due to empty returns)
      Parameters:
      matches - The ActionResult containing matches to process. The matches within this object are modified by setting their visual content and scene references.

    • setMat

      public void setMat(ActionResult matches)
      Captures visual content from the specified scene for all matches.

      This method extracts the image data (Mat/BufferedImage) from the scene at the regions defined by each match. The scene to use is determined by the sceneToUseForCaptureAfterFusingMatches setting in ActionConfig, which allows capturing from different scenes than where the match was originally found.

      This flexibility is useful when:

      • Matches are found in one scene but content needs to be captured from another
      • Multiple scenes represent different states of the same UI element
      • Post-processing requires content from a specific scene index

      Side Effects: Each match in the ActionResult is modified by:

      • Setting its scene reference to the specified scene
      • Capturing and storing the image content from that scene
      Parameters:
      matches - The ActionResult containing matches to process. The match list is modified by setting scene references and capturing images.

    • setText

      public void setText(ActionResult matches)
      Extracts text content from match regions using OCR.

      Text extraction is performed conditionally based on:

      • Whether the match already has text (avoids redundant OCR)
      • The action type and find configuration
      • Whether match fusion was applied (which removes original text)

      The method delegates the actual OCR operation to the MockOrLive service, which handles the difference between mock testing and live execution environments. Text extraction is computationally expensive, so it's only performed when necessary.

      Side Effects: Each match may be modified by setting its text content based on OCR results from the match region.

      Parameters:
      matches - The ActionResult containing matches to process. Matches without text or those meeting reset criteria will have text extracted and set via OCR.