Wednesday 20 May 2009

"Pick up a big red block", revisited

OK, I did some more work on SHRD2, and, as of a few minutes ago, I managed to speak a sentence and get it turned into a logic-like representation. Here's a slightly edited trace (the less interesting parts of the output have been removed). I did indeed say "pick up a big red block"!

>> RECOGNISE
(Take next loop input from live speech)

Recognised: recognition_succeeded([rec_result(62,pick up a big red block,...)

Old state: []
LF: [[imp,
form(imperative,
[[pick_up,term(pro,you,[]),term(a,block,[[size,big],[color,red]]),up]])]]
Dialogue move: [[imp,
quant(pro, A, [[you,A]],
quant(a, B,
[[block,B],[size,B,big],[color,B,red]],
imperative(quant(event_exists,C,[[pick_up,C,A,B,up]],true))))]]
Abstract action: say(i_dont_understand,present)
Concrete action: tts(sorry, I don't understand)
New state: []

Dialogue processing time: 0.01 seconds

Obviously, there's plenty more left to do before we have anything resembling a complete system. In particular, there is as yet no reference resolution or dialogue management, so it can't react to the commands and questions in any way. But, all things considered, I think we're making reasonable progress.

No comments: