Skip to content

[Feature]: Add configurable report screenshot retention and image encoding policies #2833

Description

@pengal

What problem does this feature solve?

Midscene reports may contain a large number of high-resolution screenshots,
especially during long-running mobile or desktop automation sessions.

The existing html-and-external-assets output format separates screenshots from
the HTML file, but it does not reduce:

  • the number of screenshots retained in a report;
  • the total report asset size;
  • unnecessary screenshots from actions that are not useful for human review.

For some use cases, users only need screenshots for specific actions, such as
Tap. Screenshots from planning, querying, input, scrolling, or other actions
do not always need to be persisted in the final report.

Users may also only need a smaller JPEG asset for human inspection instead of
the original high-resolution PNG/JPEG bytes.

This should be a Core report-generation capability, independent of Studio and
independent of screenshots used for AI perception.

Required report policies:

  1. Screenshot retention:

    • retain all screenshots;
    • retain no screenshots;
    • retain screenshots only for selected action subtypes.
  2. Report image persistence:

    • preserve the original image bytes;
    • encode persisted report images as JPEG;
    • configure JPEG quality.

The default behavior must remain unchanged for backward compatibility.

What does the proposed API look like?

Extend AgentOpt with report-specific options:

export type ReportScreenshotFilter =
  | { mode: 'all' }
  | { mode: 'none' }
  | { mode: 'actions'; actions: string[] };

export type ReportImageFormat = 'original' | 'jpeg';

export interface AgentOpt {
  /**
   * Select which action screenshots are retained in generated reports.
   * Action names match `ExecutionTask.subType`, such as `Tap` or `Input`.
   *
   * @default { mode: 'all' }
   */
  reportScreenshotFilter?: ReportScreenshotFilter;

  /**
   * Encoding used when report screenshots are persisted.
   *
   * @default 'original'
   */
  reportImageFormat?: ReportImageFormat;

  /**
   * JPEG quality used when `reportImageFormat` is `jpeg`.
   * Valid range: 1-100.
   *
   * @default 80
   */
  reportImageQuality?: number;
}



example:

const agent = new AndroidAgent(device, {
  outputFormat: 'html-and-external-assets',
  reportScreenshotFilter: {
    mode: 'actions',
    actions: ['Tap'],
  },
  reportImageFormat: 'jpeg',
  reportImageQuality: 80,
});

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions