AppTek’s call center blind evaluation set

AppTek’s call center evaluation data consists of recordings (via Zoom or other on-line platforms) of simulated call center calls. These are calls between two experienced call center agents, where one person plays a customer, and the other person plays the agent. The calls are unscripted, i.e. the two speakers discuss a certain topic spontaneously and freely, without reading from a script. The topics of the calls are mostly customer purchases of goods or services for home or office, in Retail, Finance, Hospitality, Entertainment, Agriculture, Delivery, and other areas.

The evaluation data contains multiple calls, each lasting around 10-15 minutes and saved in a separate WAV file. A range of English accents is represented in the data, including US, Canada, India, China, and other accents.

The calls are transcribed verbatim in English (transcriptions will not be provided as part of the IWSLT campaign) and translated into:

  • Simplified Chinese
  • Japanese
  • German
  • Latin American Spanish

Speaker changes are marked in the transcription and translation, so that the quality of automatic transcription/translation can be potentially evaluated separately for each of the two speakers.

Here are some examples of the transcribed speech from the data:

Example 1:

SPK1: Hello, and welcome to The Retail Company. My name is Julia. How can I help you?

SPK2: Hi, I’m just calling to check and see #uh if you have the winter collection #um from Rio available yet #uh for purchase online.

SPK1: Let me check that for you.

SPK1: Yep, it looks like it’s available for purchase online.

SPK2: #um And and do you know how long it will be available for?

SPK1: Yes, it is available for quite some time because it is the winter collection and winter is just getting started. [breath] Let me check how long it is scheduled to be open for.

Example 2:

SPK2: #um, OK, OK. #um, What about the these energy savings plan you mentioned, ma’am? #um, How do they work?

SPK1: Usually we take forty-eight hours and it includes your energy score, the potential savings and also your personalized recommendations.

SPK2: Oh, sounds really [noise] #um, that sounds really good to [noise], I like that.

SPK2: Can I also ask how billing and payments work for all these? [noise]

Example 3:

SPK1: So you want, so for every one hundred dollars, we cover five thousand.

SPK1: So does that sound like it covers your chandelier, darling?

SPK2: Not a single five thousand. Unfortunately I have expensive taste and most of the [breath] stones on here are crystal.

SPK1: [noise] Oh, I see.

For any questions, please e-mail Evgeny Matusov ematusov [-at-] apptek.com