This study investigates how ASR errors propagate through voice agent systems, affecting task completion. Controlled ASR errors were injected into LLM inputs to measure degradation across various tasks. Insertion errors were found to have a significant impact, while deletion errors showed robustness.
Key findings
Insertion errors disproportionately impact downstream reasoning in voice agents.
Deletion errors show surprising robustness with negligible impact on task completion.
Parameter extraction is identified as the primary bottleneck in voice agent pipelines.
Optimizing ASR to reduce insertion errors may improve voice agent reliability.
Limitations & open questions
The study focuses on specific error types and may not cover all real-world ASR error scenarios.
Further research is needed to generalize findings across different voice agent architectures.