Interlingua corpora for multiple domains

Following a discussion with Pierrette last week, I have added two more MedSLT Interlingua corpora, for the chest pain and abdominal pain domains. I've also added all the associated config files, scripts etc for the currently relevant language pairs (EngInt, JapInt, IntEng, IntFre and IntJap), so it should now possible to do systematic interlingua-centered development for all three domains. I have only built AFF versions, since we're planning to retire the linear formalism soon.

The naming conventions are the usual ones. I hopefully managed to check everything in, but let me know if files that you expected to find are missing. Pierrette should at some point do some work tidying up IntFre and FreInt and Yukie should do the same for IntJap and JapInt. Further down the line, we should really add coverage for these domains in the missing languages.

Interlingua corpora

Over the last few months, we have been moving MedSLT development towards a new way of doing things, which is based on the idea of an "Interlingua corpus". We present the basic picture in our LREC 2008 paper, but that's already somewhat out of date, and doesn't give any low-level details.

We now have four interlingua corpora, representing the cross-produce of {linear, AFF} x {plain, combined}. The linear/AFF distinction is concerned with the type of semantics used. "Linear" is the old MedSLT semantics; AFF semantics is explained in the paper by Pierrette, Beth Ann, Yukie and myself which has just been accepted for COLING 2008, and which will soon be appearing on the Geneva website.

The plain/combined distinction says what information has been incorporated in the corpus. The "plain" corpus is created by merging results of translating FROM each source language into interlingua, so each interlingua form lists the source language results that translate into it. The "combined" corpus contains all the information in the "plain" corpus, plus also the results of translating TO each target language.

At the moment, we use the plain corpus for developing translations rules that go from Interlingua to target languages. The combined corpus is used for creating help resources.

All the scripts used to build interlingua corpora are referenced in $MED_SLT2/Interlingua/scripts/Makefile.

Building help resources from the combined interlingua corpus (2)

Considerable progress on this task today:

1. I've added French to the AFF interlingua corpora, including the New York material as requested by Pierrette. The corpora are remade and checked in.
2. The help resources for Eng and Ara (the languages where we have help class definitions) are now made from the combined interlingua corpus. A separate help file is made for each of the six pairs EngAra, EngFre, EngJap, AraEng, AraFre, AraJap, reflecting the different levels of coverage. You can make the help resources for all of these pairs by doing 'make help_resources' in $MED_SLT2 (i.e. at the top level in the MedSLT directory), and it only takes a few minutes.
   

Building help resources from the combined interlingua corpus

I've just checked in code that allows us to build Prolog help resources from the combined interlingua corpus in multi-lingual translation applications. This will make it much easier to integrate construction of help resources into the MedSLT build - it should now be almost trivial.

I'm currently remaking the interlingua corpus (I have had to change the format a little), and should be able to check in all the relevant MedSLT stuff later this evening.

Flagging ambiguity in interlingua checking

I've just checked in code to catch cases where interlingua is
ambiguous, in the sense of generating multiple different surface
strings in the interlingua grammar. This is most likely to occur in
AFF, when the to-interlingua rules are underconstrained and the
interlingua is only partially instantiated. The following Japanese
-> Interlingua example in MedSLT illustrates:

>> doko ga itami masu ka

Source: doko ga itami masu ka
Target: WH-QUESTION pain be where PRESENT ACTIVE
Other info:
n_parses = 1
parse_time = 0.297
source_representation = [null=[path_proc,itamu], null=[tense,present],
                      null=[utterance_type,question], subject=[body_part,doko]]
source_discourse = [null=[utterance_type,question], subject=[body_part,doko],
                 null=[tense,present], null=[path_proc,itamu]]
resolved_source_discourse = [null=[utterance_type,question], subject=[body_part,doko],
                          null=[tense,present], null=[path_proc,itamu]]
resolution_processing = trivial
interlingua = [loc=[loc,where], arg1=[secondary_symptom,pain], null=[tense,present],
            null=[utterance_type,whq], null=[verb,be], null=[voice,active]]
interlingua_surface = WH-QUESTION pain be where PRESENT ACTIVE
other_interlingua_surface = [WH-QUESTION pain be above-loc where PRESENT ACTIVE,
                          WH-QUESTION pain be around-loc where PRESENT ACTIVE,
                          WH-QUESTION pain be between-loc where PRESENT ACTIVE,
                          WH-QUESTION pain be in-loc where PRESENT ACTIVE,
                          WH-QUESTION pain be under-loc where PRESENT ACTIVE]