Skip to main content
Version: 2.3.0

Using the Report

Sections​

  • Report
    • Summary
    • ASR Transcriptions
    • NLU Overall
    • NLU Intents
    • NLU Confusion Matrix
    • NLU Entities
  • Data
    • Raw table
  • Config
    • Test Run Settings

Report​

Summary​

  • Test Execution Status
  • Word Error Rate
    • Overall Transcription Accuracy
      • Expected words
      • Correct
      • Substitution
      • Insertion
      • Deletion
      • Word Error Rate
  • NLU Error Rate
    • Overall NLU Accuracy
      • Samples
      • Passed
      • Failed
      • NLU Error Rate
  • NLU Intent Error Rate
    • Overall Intent Accuracy
      • Samples
      • Passed
      • Failed
      • Intent Error Rate
  • NLU Entity Error Rate
    • Overall Slot Accuracy
      • Expected
      • Passed
      • Failed
      • Mention Error Rate
  • NLU Model Statistical Error Rate
  • NLU Model Grammar Error Rate

SUMMARY

Transcriptions​

  • Per-Intent Transcription Accuracy
    • WER
    • Bar Chart
    • Most Frequent Transcription Errors

ASR

NLU​

  • Per-Intent NLU Accuracy
    • NLU Error Rate
    • Bar Chart
  • Statistical Model Accuracy
  • Grammar Accuracy

NLU

Intents​

  • Per-Intent Intent Accuracy
    • Intent Error Rate
    • Bar Chart

Intents

Confusion Matrix​

  • Intent Confusion Matrix
    • Expected vs Actual

Confusion Matrix

Entity Mentions​

  • Per-Slot Accuracy
    • Mention Error Rate
    • Bar Chart
  • Per-Intent Slot Accuracy
    • Mention Error Rate
    • Bar Chart

Mentions

ROC Curve​

  • Accuracy numbers and percentages grouped into confidence categories and in-domain and out-of-domain groups
  • See ROC Curve for more information.

ROC Curve

Data​

Samples​

  • Each Row of Execution and it's associated data

Samples

Config​

Test Run Settings​

When the test is run, certain information is preserved and can be seen in the raw json.

This is visualized in the Test Run Settings tab.

Information presented is:

  • Tool Version
  • Client ID
  • Language
  • System Path to Results
  • ASR URL
  • ASR Settings
  • NLU URL
  • NLU Settings
  • Raw Data link

Samples

Raw JSON​

The data backing the report is available in full:

Sample Report