CALL-SLT corpus

For people who don't already know, we are collecting our initial corpus for CALL-SLT on a Google Docs document. If you don't already have access to this document and would like to contribute, please get a Google account and mail me your Google address.

All contributions very gratefully received - and don't worry that you might break something accidentally, Google Docs has excellent facilities for tracking revisions and if necessary reverting to earlier versions.

Here's a sample extract showing what the corpus looks like:

Fre: auriez-vous une table sur la terrasse (SVP) ?
Fre: auriez-vous une table en terrasse ?
Eng: do you have a table outside
Eng: is there a table outside
Eng: could i have a table outside
Int: POLITE REQUEST LOCATION-TABLE LOCATION-OUTSIDE [note: is "en terrasse" different from "outside"?]

Fre: auriez-vous une table près de la fenêtre (SVP) ?
Fre: auriez-vous une table à côté de la fenêtre (SVP) ?
Jap: mado gawa no seki wa ari masu ka
Eng: could we have a table by the window
Eng: do you have a table near the window
Int: POLITE REQUEST LOCATION-TABLE NEAR WINDOW

Fre: auriez-vous une table non-fumeur/fumeur (SVP)?
Fre: auriez-vous une table dans la zone non-fumeur /fumeur (SVP)?
Fre: auriez-vous une table dans la section non-fumeur/fumeur (SVP)?
Jap: kinenseki wa ari masu ka
Eng: i 'd like a table in the smoking area
Eng: i 'd like a table in the non-smoking area
Eng: smoking please
Eng: non-smoking please
Int: POLITE REQUEST LOCATION-TABLE IN SMOKING-AREA
Int: POLITE REQUEST LOCATION-TABLE IN NON-SMOKING-AREA

Fre: auriez-vous une table dans le coin (SVP) ?
Jap: oku no seki wa ari masu ka
Jap: oku no teeburu wa ari masu ka
Eng: could i have a table in the corner
Int: POLITE REQUEST LOCATION-TABLE IN CORNER

First CALL-SLT meeting

We've had our first CALL-SLT meeting, which has done a lot to clarify our immediate goals for the project. We're going to start by building a simple version of the system, constructed in such a way that it will be easy to upgrade it by successively replacing simple modules by more complex ones. Initially, we will be working in the tourism/restaurant domain, using the languages English, French and Japanese. When we have those working well enough, we'll also start on Chinese; this is a language none of us know, so it will give us intuitions about what it's like to be an elementary-level student trying to use the system to get some fluency in a language.

The initial prototype will work as follows. At each turn, the system prompts the student with a description of what they are supposed to say, formulated in a version of the Interlingua. The student will attempt to speak it in the L2 (the language they are trying to learn). The system applies speech recognition to the student's utterance, then tries to translate the result into the interlingua. Finally, it compares the translated interlingua with the one used to prompt the student, and gives them feedback on how they did. Here are more details:

• Prompting in Interlingua. In the first version, the interlingua will be shown to the student in a text-based form, using the methods we've developed under MedSLT. So for example, the system might show the student
  

  
  POLITE REQUEST TABLE 3 PERSON TIME 19:30
  

  
  expecting the student to say something like
  

  
  I would like to reserve a table for three people at seven thirty
  

  
  or whatever the equivalent is in the L2 they are using.
  
  
• As soon as we've figured out a good way to do it, we would like to be able to present the interlingua prompt in graphical form. So here, we might have a picture that could be described as
  

  
  Scene:
  Client is talking to waiter.
  Speech bubble from client.
  Inside speech bubble:
    three chairs around a restaurant table;
    large clock in background shows 19:30
  

• All speech input to the system will be logged in the usual way. We will have a registration process which allows us to associate each recorded utterance with meta-data which in particular will specify whether or not the utterance was recorded by a native speaker, and whether or not speech recognition got it right.
  
  
• When the system has compared the student's interlingua with the prompt interlingua, there are two simple ways for it to give helpful feedback. The first is to present both versions of the interlingua, highlighting the elements that are different. For instance, in the example above, if the system recognized
  

  
  Could I have a table for two people at seven thirty
  

  
  then the system would present the prompt and recognized interlinguas roughly as follows:
  

  
  POLITE REQUEST TABLE *3* PERSON TIME 19:30
  POLITE REQUEST TABLE *2* PERSON TIME 19:30
  

  
  The second way to give help will be to play an example of a native speaker saying some version of the sentence in the L2, if such an example already exists.
  
  
• The prompt selection module will have hooks allowing specification of a strategy. A simple strategy we will implement soon is to choose the prompt from a list of examples where there is a recorded example of a native speaker saying the prompt in the L2, possibly with some other constraints. This will make it easy for a teacher to create a lesson. They will first interact with the system in the L2, to create a set of recorded examples which work correctly. When the student logs on, the system will then be set to select prompts matching the teacher's examples.
  
  
• The functionality will be bundled up as a Prolog-based server, which does most of the processing, and will connect to a lightweight Java-based GUI which presents a client view. The server will initially handle two messages: (1) NEXT_EXAMPLE, returning a new interlingua prompt with associated information, and (2) RECOGNISE, prompting the student to speak, carrying out recognition, and returning the pieces of information produced by carrying out the interlingua comparison process.
  

Initial English recognizer for CALL-SLT (Part 2)

I have done some more work on the English CALL-SLT recognizer, and we now have about 170 surface words. You can order food and drink in various ways, e.g.

i would like two beers
could i have a pizza

There is language for reserving tables, e.g.

do you have a table for two at seven thirty
could i have a non-smoking table for three people

I've also added a few more things like asking for the menu and the check. If I could say all this stuff in a language I didn't already know, say Chinese, I'd really feel I'd learned something useful!

Initial English recognizer for CALL-SLT

We can now build an initial English recognizer for CALL-SLT too. The domain is the same, ordering in restaurants. The training corpus currently contains about 60 examples, about 95% of which parse. Vocabulary is about 130 surface words. Recognition is anecdotally quite good with my voice, though it will of course be more interesting to see how foreign voices do.

If you want more details, here are the corpus and the domain lexicon.

Initial French recognizer for CALL-SLT

Pierrette's now checked in some real (as opposed to placeholder) material, and I was able to compile an initial French recognizer for CALL-SLT. It covers some very basic tourist French, so far all about ordering in restaurants. Simple as it is, I was already able to use it to improve my pronunciation of "Je voudrais un verre d'eau". As everyone who's heard me speak French knows, my version of the "r" sound is terrible. Well, at least I can roll my eyes.

If people want to look at the CALL-SLT files, they're at http://callslt.cvs.sourceforge.net/viewvc/callslt/CALL-SLT/

Adding Romance to Regulus (part 2)

I've just checked in initial French files under CALL-SLT/Fre. Update CALLSLT with the -d flag to get them.

So far, the CALL-SLT files are pretty much the same as the MedSLT files they were adapted from. I'm assuming Pierrette will make the necessary changes! I checked that you can get as far as building a Nuance grammar... it all worked fine for me, but let me know if there are problems.

Adding Romance to Regulus

Following discussions with Pierrette, I've now moved the shared Romance grammar from MedSLT/Rom to the new directory Regulus/Grammar/Romance. I've also moved the domain-independent French grammar and lexicon files to Regulus/Grammar/Romance/French. This mirrors the directory structure in Jen's Germanic grammar.

I've adjusted the MedSLT config files for Fre, Spa and Cat to point to the new files. All three languages still appear to build correctly in the AFF (role-marked semantics) versions. I have not done anything with the old (linear semantics) versions, which I am now assuming are obsolete.

All three languages appeared to build fine when I tested, but Pierrette will probably want to check things more carefully. If there are problems, please me know - they should be easy to fix.

The payoff is that I will now be able to construct a config file for the French version of CALL-SLT. Coming up next.