How to edit syntax trees on the surface

How to edit syntax trees on the surface Peter Ljunglöf Dept. of Computer Science and Engineering Univ. of Gothenburg / Chalmers Univ. of Technology LOCI Workshop, London, 17 June 2011

How to edit syntax trees on the surface We are developing a GUI tool where users can move around the words in the sentence, and it automatically rearranges itself change the inflection of a word, and the other words automatically inflects insert, delete and replace words in the sentence, and it always keeps grammatical 2

About the tool The tool will be GUI only no keyboard input: it can be used by disabled people physically: e.g., cerebral palsy cognitively: e.g., dyslexia it can be used for 1st / 2nd language learning e.g., an interactive textbook it can be used for simple translation apps e.g., a phrasebook for smartphones 3

Example interaction a cat sleeps furious green idea colourless 4

Example interaction a furious cat sleeps green idea colourless 4

Example interaction a cat sleeps furious green idea colourless 4

Example interaction a cat sleeps furiously green idea colourless 4

Example interaction a idea sleeps furiously green colourless 4

Example interaction an idea sleeps furiously green colourless 4

Example interaction an colourless idea sleeps furiously green 4

Example interaction a colourless idea sleeps furiously green 4

Example interaction a the some many all colourless idea sleeps furiously green 4

Example interaction all colourless idea sleeps furiously green 4

Example interaction all colourless ideas sleeps furiously green 4

Example interaction all colourless ideas sleep furiously green 4

Example interaction colourless ideas sleep furiously green 4

Example interaction colourless green ideas sleep furiously 4

Overview 5

System overview sentences are stored as syntax trees the user performs editing operations on the surface string the editing operations are translated to constraints on the underlying tree the system finds a similar tree satisfying the constraints the underlying grammar is written in GF 6

The grammar formalism GF GF separates abstract and concrete syntax: abstract syntax = trees concrete syntax = surface strings This enables us to say that, e.g., these are similar: a green idea [NP [D a/an] [A green] [N idea]] all green ideas [NP [D all] [A green] [N idea]] green ideas [NP [D pl-ind] [A green] [N idea]] 7

Tree editing without trees Each word in the surface sentece is introduced by exactly one node in the abstract tree. when the user edits a word, we search for similar trees from the corresponding node it is not always the selected node that changes: a green idea all green ideas green ideas if the user wants to change idea to plural, the system has to change the determiner: [D a/an] [D all] or [D pl-ind] 8

Tree similarity We use tree edit distance: the number of replacements, insertions and deletions between two trees To limit the number of possibilities, we give different costs for different node replacements: Assume that [D a/an] [D pl-ind] is cheaper than [D a/an] [D all] Then a green idea is closer to green ideas than to all green ideas 9

Theory 10

GF: abstract syntax Abstract syntax consists of type declarations: cat, house : N this, all : N NP sleep : NP S in : NP PP npp : NP, PP NP spp : S, PP S The names on the left are called functions, and the ones on the right are called types 11

GF: concrete syntax Concrete syntax consists of compositional mappings from abstract terms to linearisations: cat = {sg= cat ; pl= cats } house = {sg= house ; pl= houses } this (x) = {s= this x.sg ; num=sg} all (x) = {s= all x.pl ; num=pl} Abstract functions can have multiple linearisations, depending on the surrounding context: house depends on the num of its parent 12

GF: concrete syntax Concrete syntax consists of compositional mappings from abstract terms to linearisations: cat = {sg= cat ; pl= cats } house = {sg= house ; pl= houses } this (x) = {s= this x.sg ; num=sg} all (x) = {s= all x.pl ; num=pl} sleep (x) = {s=x.s {sg= sleeps ;pl= sleep }.(x.num)} in (x) = {s= in x.s} npp (x,y) = {s=x.s y.s ; num=x.num} spp (x, y) = {s=x.s y.s} sleep depends on the num of its daughter 13

Terms and linearisations Example terms licensed by the grammar, and their corresponding linearisations: 14

Terms and linearisations Example terms licensed by the grammar, and their corresponding linearisations: sleep(all(cat)) = sleep (all (cat )) 14

Terms and linearisations Example terms licensed by the grammar, and their corresponding linearisations: sleep(all(cat)) = sleep (all (cat )) = sleep ({s= all + cat.pl; num=pl}) 14

Terms and linearisations Example terms licensed by the grammar, and their corresponding linearisations: sleep(all(cat)) = sleep (all (cat )) = sleep ({s= all + cat.pl; num=pl}) = sleep ({s= all + {sg= cat ;pl= cats }.pl; num=pl}) = sleep ({s= all + cats ; num=pl}) = = {s = all cats sleep } sleep(this(cat)) = sleep (this (cat )) 14

GF terms are syntax trees GF terms are syntax sleep trees where the nodes are labeled with npp functions, not types all in We write v for a node in a tree t, and v for its linearisation cat this house 15

Constrained linearisation We define the constrained linearisation [v]t as the strings in v that are actually used in t The linearisation of cat is: cat = {sg= cat ; pl= cats } It has the following constrained linearisations: [cat]t = cats when t = sleep(all(cat)) [cat]t = cat when t = sleep(this(cat)) 16

Tree editing without trees So, how can we edit syntax trees in a GUI where the trees are not shown? It is not always the active node that changes: a green idea all green ideas if the user edits the noun (idea), we have to change the determiner (a/an all) Solution: search for the closest (most similar) tree matching the given constraints 17

Editing operations 18

Editing operations What can the user do with a sentence? select a phrase, or increase the selection display and select from a context menu delete, move, insert, replace a word/phrase Each editing operation introduces constraints on the underlying tree and its linearisation we search for the closest tree matching the introduced constraints 19

select menu delete move insert replace Select a phrase When the user clicks a word w with node v: if v is unselected, select it if v is selected, increase the selection: select an ancestor node such that more words are included When we say the user clicks a node v, we mean that the user clicks a word w covered by the selected node v. 20

select menu delete move insert replace Display a context menu If the user context-clicks the node v: search for nearby trees t such that [v]t [v]t display them in a menu if the user selects a menu item, replace the current tree with the new tree t 21

select menu delete move insert replace Delete a phrase If the user deletes the selected node v: search for the closest tree t such that either v t or [v]t is empty 22

select menu delete move insert replace Move or insert a phrase If the user moves a node v to another position, between words w1 and w2, in the sentence: search for the closest tree t such that [v1] [v] [v2], where v1 and v2 are the nodes covering w1 and w2 Similar when the user inserts a new phrase but then v denotes a new node 23

select menu delete move insert replace Replace a phrase If the user drops a new word/phrase v onto an existing node v: search for the closest tree t such that all descendants of v are deleted all descendants of v are included the children of v should be children of v the label of v should be the label of v 24

Final remarks 25

Status Currently we are working on a web demo: it is implemented in Javascript, the GF grammar is compiled into JS it will work on both traditional computers and touchscreen tablets it will work with one single sentence (not full texts), and one single language it is only partly working yet 26

Summary the main idea is that the user s editing operations are translated to constraints on the underlying syntactical structure the idea is theory-neutral we are using GF, but most grammar formalisms probably work it probably works best on limited domains, otherwise we have the risk of too many similar structures which can make the system act unituitive 27

Possible applications Language learning/training: interactive textbook 1st and 2nd language learning Translation: phrasebook for smartphones Dialogue modeling? repair strategies? 28

Application: interactive textbook A traditional textbook is divided into chapters: each chapter introduces a new concept (e.g., NP congruence, fronting) there is an associated exemplifying text We do exactly the same, but add interaction: the chapters are grammar modules the user can experiment with the text, change, move, insert, delete and the text will always keep grammatical 29

The interactive textbook tool The tool will be GUI only no keyboard input: it can be used by disabled people physically: e.g., cerebral palsy cognitively: e.g., dyslexia it can be used for both 1L and 2L learning it can use multilingual grammars e.g., Swedish French (2L) e.g., Swedish Symbols (1L for disabled) 30