Statistical Machine Translation Lecture 5. Decoding with Phrase-Based Models

Size: px

Start display at page:

Download "Statistical Machine Translation Lecture 5. Decoding with Phrase-Based Models"

Ashlynn Ophelia Fowler
6 years ago
Views:

1 p. Statistical Machine Translation Lecture 5 Decoding with Phrase-Based Models Stephen Clark based on slides by Phillip Koehn

2 p. Statistical Machine Translation p Components: Translation model, language model, decoder foreign/english parallel text English text statistical analysis Translation Model statistical analysis Language Model Decoding Algorithm

3 p. Phrase-Based Translation p Morgen fliege ich nach Kanada zur Konferenz Tomorrow I will fly to the conference in Canada Foreign input is segmented in phrases any sequence of words, t necessarily linguistically motivated Each phrase is translated into English Phrases are reordered

4 p. Phrase Translation Table p Phrase Translations for den Vorschlag : English φ(e f) English φ(e f) the proposal the suggestions s proposal the proposed a proposal the motion the idea the idea of this proposal the proposal, proposal its proposal of the proposal it the proposals

5 p. Decoding Process p Maria dio una bofetada a la bruja verde Build translation left to right select foreign words to be translated

6 p. Decoding Process p Maria dio una bofetada a la bruja verde Mary Build translation left to right select foreign words to be translated find English phrase translation add English phrase to end of partial translation

7 p. Decoding Process p Maria dio una bofetada a la bruja verde Mary Build translation left to right select foreign words to be translated find English phrase translation add English phrase to end of partial translation mark foreign words as translated

8 p. Decoding Process p Maria dio una bofetada a la bruja verde Mary did t One to many translation

9 p. Decoding Process p Maria dio una bofetada a la bruja verde Mary did t slap Many to one translation

10 p. 1 Decoding Process p Maria dio una bofetada a la bruja verde Mary did t slap the Many to one translation

11 p. 1 Decoding Process p Maria dio una bofetada a la bruja verde Mary did t slap the green Reordering

12 p. 1 Decoding Process p Maria dio una bofetada a la bruja verde Mary did t slap the green witch Translation finished

13 p. 1 Translation Options p Maria dio una bofetada a la bruja verde Mary t give a slap to the witch green did t a slap by green witch slap to the did t give to the slap the witch Look up possible phrase translations many different ways to segment words into phrases many different ways to translate each phrase

14 p. 1 Hypothesis Expansion p Maria dio una bofetada a la bruja verde Mary t give a slap to the witch green did t a slap by green witch slap to the did t give to the slap the witch e: f: p: 1 Start with empty hypothesis e: English words f: foreign words covered p: probability 1

15 p. 1 Hypothesis Expansion p Maria dio una bofetada a la bruja verde Mary t give a slap to the witch green did t a slap by green witch slap to the did t give to the slap the witch e: f: p: 1 e: Mary f: * p:.534 Pick translation option Create hypothesis e: add English phrase Mary f: first foreign word covered p: probability 0.534

16 p. 1 A Quick Word on Probabilities p Not going into detail here, but... Translation Model phrase translation probability p(mary Maria) reordering costs phrase/word count costs... Language Model uses trigrams: p(mary did t) = p(mary <s>) * p(did Mary,<s>) * p(t Mary did)

17 p. 1 Hypothesis Expansion p Maria dio una bofetada a la bruja verde Mary t give a slap to the witch green did t a slap by green witch slap to the did t give to the slap the witch e: witch f: *- p:.182 e: f: p: 1 e: Mary f: * p:.534 Add ather hypothesis

18 p. 1 Hypothesis Expansion p Maria dio una bofetada a la bruja verde Mary t give a slap to the witch green did t a slap by green witch slap to the did t give to the slap the witch e: witch f: *- p:.182 e:... slap f: *-***---- p:.043 e: f: p: 1 e: Mary f: * p:.534 Further hypothesis expansion

19 p. 1 Hypothesis Expansion p Maria dio una bofetada a la bruja verde Mary t give a slap to the witch green did t a slap by green witch slap to the did t give to the slap the witch e: witch f: *- p:.182 e: slap f: *-***---- p:.043 e: f: p: 1 e: Mary f: * p:.534 e: did t f: ** p:.154 e: slap f: *****---- p:.015 e: the f: *******-- p: e:green witch f: ********* p: until all foreign words covered find best hypothesis that covers all foreign words backtrack to read off translation

20 p. 2 Hypothesis Expansion p Maria dio una bofetada a la bruja verde Mary t give a slap to the witch green did t a slap by green witch slap to the did t give to the slap the witch e: witch f: *- p:.182 e: slap f: *-***---- p:.043 e: f: p: 1 e: Mary f: * p:.534 e: did t f: ** p:.154 e: slap f: *****---- p:.015 e: the f: *******-- p: e:green witch f: ********* p: Adding more hypothesis Explosion of search space

21 p. 2 Explosion of Search Space p Number of hypotheses is exponential with respect to sentence length Decoding is NP-complete [Knight, 1999] Need to reduce search space risk free: hypothesis recombination risky: histogram/threshold pruning

22 p. 2 Hypothesis Recombination p p=0.092 p=1 Mary p=0.534 did t give p=0.092 did t p=0.164 give p=0.044 Different paths to the same partial translation

23 p. 2 Hypothesis Recombination p p=0.092 p=1 Mary p=0.534 did t give p=0.092 did t p=0.164 give Different paths to the same partial translation Combine paths drop weaker hypothesis keep pointer from worse path

24 p. 2 Hypothesis Recombination p p=1 Joe Mary p=0.092 p=0.534 did t give did t give p=0.017 p=0.092 did t p=0.164 give Recombined hypotheses do t have to match completely No matter what is added, weaker path can be dropped, if: last two English words match (matters for language model) foreign word coverage vectors match (affects future path)

25 p. 2 Hypothesis Recombination p p=1 Joe Mary p=0.092 p=0.534 did t give did t give p=0.092 did t p=0.164 give Recombined hypotheses do t have to match completely No matter what is added, weaker path can be dropped, if: last two English words match (matters for language model) foreign word coverage vectors match (effects future path) Combine paths

26 p. 2 Pruning p Hypothesis recombination is t sufficient Heuristically discard weak hypotheses Organize Hypothesis in stacks, e.g. by same foreign words covered same number of foreign words covered same number of English words produced Compare hypotheses in stacks, discard bad ones histogram pruning: keep top n hypotheses in each stack (e.g., n=100) threshold pruning: keep hypotheses that are at most α times the cost of best hypothesis in stack (e.g., α = 0.001)

27 p. 2 Hypothesis Stacks p Organization of hypothesis into stacks here: based on number of foreign words translated during translation all hypotheses from one stack are expanded expanded Hypotheses are placed into stacks

28 p. 2 Comparing Hypotheses p Comparing hypotheses with same number of foreign words covered Maria dio una bofetada a la bruja verde e: Mary did t f: ** p: better partial translation e: the f: -----**-- p: covers easier part --> lower cost Hypothesis that covers easy part of sentence is preferred Need to consider future cost of uncovered parts

CSE 517 Natural Language Processing Winter 2013

CSE 517 Natural Language Processing Winter 2013 Phrase Based Translation Luke Zettlemoyer Slides from Philipp Koehn and Dan Klein Phrase-Based Systems Sentence-aligned corpus Word alignments cat chat 0.9