Video Summary Accuracy Analyzer#

GitHub Readme

The Video Summary Accuracy Analyzer is an evaluation tool that measures the quality of a generated video summary by comparing it against a ground-truth reference summary. It is accessible through both a web-based dashboard and a REST API, enabling integration into automated evaluation workflows.

Note

This tool is designed solely for summary evaluation. It does not process video files, generate reference summaries, or perform video summarization using VLMs, video summarization tools, or pipeline workflows.

Key Metrics#

  • Lexical Similarity: Assesses word-level overlap and coverage between the generated summary and the reference summary.

  • Semantic Similarity: Evaluates how well the generated summary preserves the meaning and context of the reference summary.

  • Factual Consistency: Identifies alignment of key facts, entities, and events between the generated and reference summaries.

  • Temporal Coherence: Measures the accuracy of event ordering and the overall narrative flow.

Next Steps#

  • Get Started: Configure, deploy, and use the dashboard and API with Docker Compose.

  • Build from Source: Build the backend and UI container images.