We've enhanced the Statistics section in EasySLR by introducing Precision and F1 Score metrics for both human reviewers and AI-assisted screening. These additions provide deeper insights into screening performance and help teams evaluate the quality and consistency of review decisions across their projects.

By making these metrics available for both human and AI reviewers, EasySLR enables teams to compare performance, monitor screening quality, and better understand the effectiveness of their review workflows.

What's New?

The Quality dashboard in Statistics section now includes two additional AI performance metrics:

• Precision

• F1 Score

These metrics are calculated for both:

• Human reviewers

• AI reviewers

They are available alongside the existing screening statistics, allowing project owners to evaluate reviewer performance using widely accepted machine learning and information retrieval measures.

Precision

Precision measures how many of the articles marked as included by a reviewer (human or AI) were ultimately considered relevant when compared with the final review decisions.

A higher Precision score indicates fewer false-positive inclusion decisions, resulting in more accurate screening outcomes.

This metric helps teams understand how confidently both human reviewers and AI identify relevant studies during the screening process.

F1 Score

The F1 Score provides a balanced measure of screening performance by combining both Precision and Recall into a single metric.

Rather than evaluating only one aspect of reviewer performance, the F1 Score assesses how effectively both human reviewers and AI balance:

• Identifying relevant studies (Recall)

• Avoiding incorrect inclusions (Precision)

This provides a more comprehensive view of overall screening quality.

Why This Matters

Evaluating both human and AI performance using standardised metrics provides greater transparency into the screening process.

While Recall measures how effectively relevant studies are identified, Precision measures the accuracy of inclusion decisions. The F1 Score combines both measures into a single indicator, enabling teams to objectively assess screening performance regardless of whether decisions are made by human reviewers or AI.

This is particularly valuable for teams adopting AI-assisted workflows, comparing reviewer performance, or monitoring consistency across multiple reviewers.

Key Benefits

Comprehensive Performance Evaluation

Evaluate the screening quality of both human reviewers and AI using recognised performance metrics.

Better Reviewer & AI Validation

Understand how closely reviewer decisions align with final outcomes and identify opportunities to improve screening consistency.

Improved Quality Monitoring

Compare reviewer performance across projects, monitor AI effectiveness, and identify areas requiring additional training or workflow refinement.

Increased Transparency

Gain clearer visibility into the accuracy and effectiveness of both human and AI screening decisions, supporting more informed quality assurance and adoption of AI-assisted workflows.

Common Use Cases

Researchers and project owners can now:

• Evaluate the screening quality of both human reviewers and AI

• Compare reviewer performance across projects

• Monitor AI performance alongside human reviewers

• Assess improvements after refining screening protocols or reviewer training

• Validate AI-assisted workflows before broader adoption

• Report standardised reviewer and AI performance using internationally accepted evaluation metrics

Summary

The addition of Precision and F1 Score to the Statistics dashboard provides a more complete and transparent view of screening performance for both human reviewers and AI.

By evaluating reviewer accuracy and consistency using standardised metrics, EasySLR helps research teams monitor screening quality, improve workflow efficiency, and make more informed decisions when conducting AI-assisted systematic reviews.