Sunday 24 May 2009

First version of paraphrase grammar for SHRD2

I've just added a first version of a paraphrase grammar for SHRD2, where it tries to realize the scoped LFs in "logic English". Text example below. It doesn't do much more than this yet, but now that I have the basic structure working I'm hoping that it will be easy to add coverage. Planning to continue with this later today!

>> pick up a big red block

Old state: []
LF: [[imp,
form(imperative,
[[pick_up,term(pro,you,[]),term(a,block,[[size,big],[color,red]]),up]])]]
Intermediate 1: [[imp,
scoping_unit([modal, imperative,
term(event_exists, A,
[[pick_up, A, term(pro,B,[[you,B]]),
term(a,C,[[block,C],[size,C,big],[color,C,red]]), up]])])]]
Intermediate 2: [[imp,
quant(pro, A, [[you,A]],
quant(a, B,
[[block,B],[size,B,big],[color,B,red]],
imperative(quant(event_exists,C,[[pick_up,C,A,B,up]],true))))]]
Intermediate 3: [[imp,
quant(pro, A, [[you,A]],
quant(a, B,
[[block,B],[size,B,big],[color,B,red]],
imperative(quant(event_exists,C,[[pick_up,C,A,B]],true))))]]
Dialogue move: [[imp,
quant(exist, A, [[you,A]],
quant(exist, B,
[[block,B],[size,B,big],[color,B,red]],
imperative(quant(exist,C,[[pick_up,C,A,B]],true))))]]
Paraphrase: COMMAND there is an A SUCH THAT you are A AND there is a B SUCH THAT B is a block AND B is big AND B is red AND MAKE IT TRUE THAT there is a C SUCH THAT C is that A picks up B
Abstract action: say(i_dont_understand,present)

Thursday 21 May 2009

Multiple processing stages in dialogue top-level

When building dialogue applications, it's quite often the case that there are multiple stages in the process of converting the LF (the thing that comes out of the recognizer) into a dialogue move. I've just added some hooks so that you can pass back the intermediate levels of representation as an optional extra argument to lf_to_dialogue_move, and display them. This is useful for debugging.

Here's a text example from SHRD2:

>> is the large red block in the box

Old state: []
LF: [[ynq,
form(present,
[[be, term(the_sing,block,[[size,large],[color,red]]),
[in_loc,term(the_sing,box,[])]]])]]
Intermediate 1: [[ynq,
scoping_unit(term(event_exists, A,
[[be, A,
term(the_sing, B,
[[block,B],[size,B,large],[color,B,red]]),
[in_loc,term(the_sing,C,[[box,C]])]],
[tense,A,present]]))]]
Intermediate 2: [[ynq,
quant(the_sing, A, [[block,A],[size,A,large],[color,A,red]],
quant(the_sing, B, [[box,B]],
quant(event_exists, C,
[[be,C,A,[in_loc,B]],[tense,C,present]], true)))]]
Dialogue move: [[ynq,
quant(the_sing, A, [[block,A],[size,A,big],[color,A,red]],
quant(the_sing, B, [[box,B]],
quant(event_exists,C,[[be_in_loc,C,A,B],[tense,C,present]],true)))]]
Abstract action: say(i_dont_understand,present)
Concrete action: tts(sorry, I don't understand)
New state: []

Intermediate 1 is after addition of variables; Intermediate 2 is after scoping; and Dialogue move is after rewriting of lexical predicates. This last step is still very primitive.

Wednesday 20 May 2009

"Pick up a big red block", revisited

OK, I did some more work on SHRD2, and, as of a few minutes ago, I managed to speak a sentence and get it turned into a logic-like representation. Here's a slightly edited trace (the less interesting parts of the output have been removed). I did indeed say "pick up a big red block"!

>> RECOGNISE
(Take next loop input from live speech)

Recognised: recognition_succeeded([rec_result(62,pick up a big red block,...)

Old state: []
LF: [[imp,
form(imperative,
[[pick_up,term(pro,you,[]),term(a,block,[[size,big],[color,red]]),up]])]]
Dialogue move: [[imp,
quant(pro, A, [[you,A]],
quant(a, B,
[[block,B],[size,B,big],[color,B,red]],
imperative(quant(event_exists,C,[[pick_up,C,A,B,up]],true))))]]
Abstract action: say(i_dont_understand,present)
Concrete action: tts(sorry, I don't understand)
New state: []

Dialogue processing time: 0.01 seconds

Obviously, there's plenty more left to do before we have anything resembling a complete system. In particular, there is as yet no reference resolution or dialogue management, so it can't react to the commands and questions in any way. But, all things considered, I think we're making reasonable progress.

Sunday 17 May 2009

CAT and FEAT commands

One of the good things about doing some grammar development on SHRD2 is that it suggests ideas for new development environment functionality. I got tired of consulting the grammar every time I needed to find out what features a category had, or what possible values a feature could take. So I added a couple of new commands called CAT and FEAT, which give you those pieces of information. Here's an example:

>> CAT p
(Display information for specified category)

Features for category "p": [def,obj_sem_n_type,postposition,sem,sem_p_type,sem_pp_type]

--- Performed command CAT p, time = 0.00 seconds

>> FEAT sem_p_type
(Display information for specified feature)

Feature values for feature "sem_p_type": [[back,down,none,normal,off,onoff,over,up,updown]]

--- Performed command FEAT sem_p_type, time = 0.00 seconds

SHRD2

I was talking with some people about Terry Winograd's book Understanding Natural Language, and how inspiring I found the SHRDLU system when I first read about it as an undergraduate. They asked me what the current equivalent would be. It bothered me that I couldn't come up with anything, and that no one really seemed to be building this kind of system any more.

Well... so why not do something about it? Stage 1 is to build a speech-enabled, and hopefully rather more robust reconstruction of SHRDLU, using Regulus. Some initial stuff is already checked in under $REGULUS/Examples/SHRD2. When we've satisfied ourselves that we can handle the Blocks World, Stage 2 will be to define a new and more ambitious domain - something which will hopefully demonstrate that there has in fact been significant progress since the 70s.

I'll be posting more about this soon. So far, SHRD2 has already turned up some important holes in the general English grammar, which I'm working on fixing.

Showing missing vocabulary in EBL_TREEBANK

A trivial but rather useful little feature I just added: when you run EBL_TREEBANK, you now get missing vocabulary displayed for relevant sentences. I don't know why I didn't do this years ago. Here's an example. To get the new functionality, you just need to update Regulus from CVS, nothing needs to be remade.

>> EBL_TREEBANK
(Parse all sentences in current EBL training set into treebank form)

--- Read parsing history file (114 records) d:/cygwin/home/speech/regulus/examples/shrd2/generated/shrd2_parsing_history.pl
--- Incremental treebanking switched off, not trying to convert treebank

Parsing corpus data in d:/cygwin/home/speech/regulus/examples/shrd2/corpora/shrdlu_corpus.pl:
..
*** Parsing failed for: "find a block which is taller than the one you are holding and put it into the box", line 2
...
*** Parsing failed for: "is at least one of them narrower than the one which i told you to pick up", line 6
Words not in current vocabulary: [told]
.....
*** Parsing failed for: "will you please stack up both of the red blocks and either a green cube or a pyramid", line 12
Words not in current vocabulary: [either]
...