Enhancing the Usability and Performance of NESPOLE! — a Real-World Speech-to-Speech Translation System

Abstract

Nespole! is a speech-to-speech machine translation project designed to provide fully functional speech-to-speech capabilities within real-world settings of common users involved in e-commerce applications. The project is a collaboration between three European research groups (IRST in Trento, Italy; ISL at Universit?t Karlsruhe (TH); and CLIPS at Universit? Joseph Fourier in Grenoble, France), one US research group (ISL at Carnegie Mellon University in Pittsburgh, PA) and two industrial partners (APT; Trento, Italy ?the Trentino provincial tourism board, and AETHRA; Ancona, Italy ?a tele-communications company). The project is funded jointly by the European Commission and the US NSF. Over the past year, we have developed a fully functional showcase of the Nespole! system within the domain of travel and tourism, and have significantly improved system performance and usability based on a series of studies and evaluations with real users. Our experience has shown that improving translation quality is only one of several important issues that must be addressed in achieving a practical real-world speech-to-speech translation system. This paper describes how we tackled these issues and evaluates their effect on system performance and usability. We focus on three main issues: (1) a study on the usage and utility of multi-modality in the context of multi-lingual communication; (2) assessing system performance under various network traffic conditions and architectural configurations; and (3) an end-to-end evaluation of the demonstration system.