Update on SHRD2 paraphrasing

I extended the paraphrasing capabilities for SHRD2, and we now get paraphrases for about 45% of the 180-ish examples in the current corpus. Beth Ann and I did a little experiment for the paper we presented at the SETQA-NLP workshop in Colorado last week. We each tried judging the corpus examples for correctness, using both the paraphrases and the underlying representations. In order to minimize learning effects, we permuted the order in which we did things.

Even though the "logic English" paraphrases seem very similar to the scoped logical representations, it in fact turns out that judging paraphrases is a lot faster. Even for me, knowing all the representations from having worked on them, judging paraphrases took 29 minutes, against 22 minutes for judging structures. For Beth Ann, who didn't know the datastructures previously, paraphrases were more than twice as quick. Part of the payoff comes from the fact that the paraphrase grammar acts as a filter; most ill-formed structures produce no paraphrase, hence don't need to be judged at all when paraphrases are used.

One person in the workshop audience said he was pleased to see a paper about software engineering which actually contained an experiment! Beth Ann was clearly right to insist that we do this, and work out the methodology.

First version of paraphrase grammar for SHRD2

I've just added a first version of a paraphrase grammar for SHRD2, where it tries to realize the scoped LFs in "logic English". Text example below. It doesn't do much more than this yet, but now that I have the basic structure working I'm hoping that it will be easy to add coverage. Planning to continue with this later today!

>> pick up a big red block

      Old state: []
             LF: [[imp, 
                   form(imperative, 
                        [[pick_up,term(pro,you,[]),term(a,block,[[size,big],[color,red]]),up]])]]
 Intermediate 1: [[imp, 
                   scoping_unit([modal, imperative, 
                                 term(event_exists, A, 
                                      [[pick_up, A, term(pro,B,[[you,B]]), 
                                        term(a,C,[[block,C],[size,C,big],[color,C,red]]), up]])])]]
 Intermediate 2: [[imp, 
                   quant(pro, A, [[you,A]], 
                         quant(a, B, 
                               [[block,B],[size,B,big],[color,B,red]], 
                               imperative(quant(event_exists,C,[[pick_up,C,A,B,up]],true))))]]
 Intermediate 3: [[imp, 
                   quant(pro, A, [[you,A]], 
                         quant(a, B, 
                               [[block,B],[size,B,big],[color,B,red]], 
                               imperative(quant(event_exists,C,[[pick_up,C,A,B]],true))))]]
  Dialogue move: [[imp, 
                   quant(exist, A, [[you,A]], 
                         quant(exist, B, 
                               [[block,B],[size,B,big],[color,B,red]], 
                               imperative(quant(exist,C,[[pick_up,C,A,B]],true))))]]
     Paraphrase: COMMAND there is an A SUCH THAT you are A AND there is a B SUCH THAT B is a block AND B is big AND B is red AND MAKE IT TRUE THAT there is a C SUCH THAT C is that A picks up B
Abstract action: say(i_dont_understand,present)