Enhanced Action Logging with Console Output and Visual Feedback
Overviewโ
The Brobot framework includes enhanced action logging that provides real-time console output and visual highlighting during automation execution. This makes debugging and development significantly easier by showing exactly what Brobot is doing at each step.
Key Featuresโ
1. Console Action Reportingโ
- Real-time feedback with visual indicators (๐, โ, โ, โ ๏ธ)
- Configurable verbosity levels (QUIET, NORMAL, VERBOSE)
- Performance warnings for slow operations
- Detailed match information including location and confidence scores
2. Visual Highlightingโ
- Green borders for successful pattern matches
- Blue borders for search regions
- Yellow ripple effects for clicks
- Red indicators for errors
- All colors and durations are configurable
3. GUI Access Detectionโ
- Automatic detection of environment issues
- Platform-specific solutions (Linux/X11, Windows, macOS)
- Clear, actionable advice for fixing problems
Quick Startโ
Basic Usageโ
The enhanced logging is automatically enabled. Just run your Brobot application and you'll see:
๐ FIND: login-button โ โ FOUND (234ms)
โโ Location: (450,320) Score: 98.5%
โ CLICK login-button (156ms)
โ FIND submit-button (2003ms)
โโ Search regions: 3 areas checked
โ ๏ธ Performance Warning: FIND took 2003ms (threshold: 1000ms)
Configurationโ
Add to your application.yml
:
brobot:
# Console output settings
console:
actions:
enabled: true
level: NORMAL # QUIET, NORMAL, VERBOSE
show-match-details: true
use-icons: true
# Visual highlighting
highlight:
enabled: true
auto-highlight-finds: true
find:
color: "#00FF00"
duration: 2.0
Using Visual Debug Profileโ
For maximum visibility during development:
java -jar your-app.jar --spring.profiles.active=visual-debug
Common Use Casesโ
1. Debugging Failed Findsโ
With VERBOSE logging, you'll see exactly where Brobot searched:
โ FIND submit-button (2003ms)
โโ Search regions: 3 areas checked
โโ Region 1: (0,0 800x600)
โโ Region 2: (800,0 800x600)
โโ Similar matches: button-disabled (85.2%)
2. Performance Optimizationโ
Identify slow operations immediately:
โ ๏ธ Performance Warning: FIND took 2003ms (threshold: 1000ms)
3. GUI Environment Issuesโ
Get immediate feedback about environment problems:
โ GUI Problem: No DISPLAY environment variable set
๐ก Possible solutions:
โข Set DISPLAY=:0 for local display
โข For SSH: use -X or -Y flag for X11 forwarding
โข For Docker: pass --env DISPLAY=$DISPLAY
โข For WSL: install and configure X server (VcXsrv, Xming)
Console Action Reportingโ
Featuresโ
The console reporter provides formatted output with:
- Visual icons for different action types (๐ for FIND, ๐ for CLICK, etc.)
- Success/failure indicators (โ/โ)
- Timing information for each action
- Match details including location and confidence scores
- Performance warnings for slow operations
Example Outputโ
๐ FIND: login-button โ โ FOUND (234ms)
โโ Location: (450,320) Score: 98.5%
๐ CLICK: login-button (156ms) โ
โจ๏ธ TYPE: "user@example.com" โ
โ FIND submit-button (2003ms)
โโ Search regions: 3 areas checked
โโ Similar matches: button-disabled (85.2%)
โ ๏ธ Performance Warning: FIND took 2003ms (threshold: 1000ms)
๐ STATE: LoginPage โ Dashboard [425ms] [SUCCESS]
Configurationโ
Configure console output through properties:
brobot:
console:
actions:
# Enable/disable console reporting
enabled: true
# Verbosity level
level: NORMAL # Options: QUIET, NORMAL, VERBOSE
# Show match details in VERBOSE mode
show-match-details: true
# Display timing information
show-timing: true
# Use colored output (requires ANSI support)
use-colors: true
# Use unicode icons
use-icons: true
# Filter which actions to report
report:
find: true
click: true
type: true
drag: true
highlight: false
# Report state transitions
report-transitions: true
# Performance thresholds (milliseconds)
performance-warn-threshold: 1000
performance-error-threshold: 5000
Verbosity Levelsโ
- QUIET: Minimal output - only errors and failures
- NORMAL: Standard output - actions with success/failure
- VERBOSE: Detailed output - includes match details, timing, and metadata
Visual Highlightingโ
Visual feedback helps you see exactly what Brobot is doing on screen.
Highlight Typesโ
-
Find Highlights (Green by default)
- Shows successful pattern matches
- Displays match confidence score (optional)
- Can flash to draw attention
-
Search Region Highlights (Blue by default)
- Shows areas being searched
- Displays region dimensions (optional)
- Helps understand search scope
-
Click Highlights (Yellow by default)
- Shows click locations
- Optional ripple effect
- Helps verify click accuracy
-
Error Highlights (Red by default)
- Shows where searches failed
- Optional cross mark overlay
- Helps identify problem areas
Configurationโ
brobot:
highlight:
# Global enable/disable
enabled: true
# Automatically highlight successful finds
auto-highlight-finds: true
# Automatically highlight search regions
auto-highlight-search-regions: false
# Find highlighting configuration
find:
color: "#00FF00" # Green
duration: 2.0 # seconds
border-width: 3 # pixels
flash: false # Enable flashing
flash-count: 2 # Number of flashes
flash-interval: 300 # milliseconds
# Search region highlighting
search-region:
color: "#0000FF" # Blue
duration: 1.0
border-width: 2
opacity: 0.3 # For filled regions
filled: false # Fill region or just border
show-dimensions: false
# Error highlighting
error:
enabled: false
color: "#FF0000" # Red
duration: 3.0
show-cross-mark: true
# Click highlighting
click:
enabled: true
color: "#FFFF00" # Yellow
duration: 0.5
radius: 20 # pixels
ripple-effect: true
GUI Access Detectionโ
The GUI access monitor automatically detects and reports environment issues that could prevent automation from working.
Common Issues Detectedโ
- Headless environment (no display)
- Missing DISPLAY variable on Linux
- X11 server not accessible
- Wayland compatibility issues
- Remote desktop limitations
- Low screen resolution
Example Outputโ
โ GUI Problem: No DISPLAY environment variable set
๐ก Possible solutions:
โข Set DISPLAY=:0 for local display
โข For SSH: use -X or -Y flag for X11 forwarding
โข For Docker: pass --env DISPLAY=$DISPLAY
โข For WSL: install and configure X server (VcXsrv, Xming)
โ GUI Problem: Running in Remote Desktop session
๐ก Possible solutions:
โข Some screen capture features may be limited
โข Consider running directly on the machine
โข Use alternative remote access tools if needed
Configurationโ
brobot:
gui-access:
# Report problems to console
report-problems: true
# Show detailed error information
verbose-errors: true
# Suggest solutions for problems
suggest-solutions: true
# Check on application startup
check-on-startup: true
# Continue despite GUI problems
continue-on-error: false
# Minimum screen resolution
min-screen-width: 800
min-screen-height: 600
Advanced Featuresโ
Custom Visual Feedbackโ
@Autowired
private EnhancedActionLogger logger;
// Create custom visual options
VisualFeedbackOptions options = VisualFeedbackOptions.builder()
.highlightFinds(true)
.findHighlightColor(Color.YELLOW)
.findHighlightDuration(5.0)
.flashHighlight(true)
.showMatchScore(true)
.build();
logger.logActionWithVisuals("FIND", target, result, options);
Profile-Based Configurationโ
Development Profileโ
# Maximum visibility
brobot:
console.actions.level: VERBOSE
highlight.auto-highlight-finds: true
highlight.auto-highlight-search-regions: true
CI/CD Profileโ
# Minimal output
brobot:
console.actions.level: QUIET
highlight.enabled: false
gui-access.continue-on-error: true
Production Profileโ
# Disabled
brobot:
console.actions.enabled: false
highlight.enabled: false
Troubleshootingโ
No Console Outputโ
- Check if enabled:
brobot.console.actions.enabled: true
- Check verbosity:
brobot.console.actions.level: NORMAL
- Check action filters:
brobot.console.actions.report.find: true
Icons Not Showingโ
Some terminals don't support unicode. Disable icons:
brobot.console.actions.use-icons: false
Visual Highlighting Not Workingโ
- Check if enabled:
brobot.highlight.enabled: true
- Verify GUI access: Run the GUI access check
- Check if in mock mode (highlights are logged but not shown visually)
Example Applicationโ
See the complete example in /examples/src/main/java/io/github/jspinak/brobot/examples/logging/EnhancedLoggingDemo.java
Run it with:
cd examples
./gradlew bootRun
Or with visual debug profile:
./gradlew bootRun --args='--spring.profiles.active=visual-debug'
Performance Impactโ
- Console output: ~1ms overhead per action
- Visual highlighting: Asynchronous, non-blocking
- GUI access check: Only on startup (configurable)
- Overall impact: < 2% in typical usage
Migration from Old Loggingโ
The new system is backward compatible. Existing code continues to work, but you can enhance it:
// Old way
actionLogger.logAction("CLICK", button, result);
// Enhanced way with visual feedback
enhancedLogger.logActionWithVisuals("CLICK", button, result,
VisualFeedbackOptions.debug());
Configuration Referenceโ
See /library/src/main/resources/brobot-visual-feedback.properties
for all available settings with documentation.
Integration with Desktop Runnerโ
The desktop runner automatically integrates console output through the ConsoleActionEventListener
. No additional configuration is needed - console output appears in both the terminal and the runner's log viewer.
Best Practicesโ
- Use profiles to configure different environments appropriately
- Enable verbose logging during development, disable in production
- Check GUI access early to catch environment issues
- Use visual feedback sparingly in automated tests to avoid delays
- Configure performance thresholds based on your application's needs
- Filter action types to reduce noise in logs
API Referenceโ
EnhancedActionLogger
- Extended logging interfaceVisualFeedbackOptions
- Visual feedback configurationConsoleActionConfig
- Console output settingsGuiAccessMonitor
- GUI environment checking
Next Stepsโ
- Enable enhanced logging in your application
- Try the visual-debug profile during development
- Customize colors and durations for your needs
- Use GUI access detection to catch environment issues early
- Monitor performance with automatic warnings