Supervised Learning of Complete Morphological Paradigms

Size: px

Start display at page:

Download "Supervised Learning of Complete Morphological Paradigms"

Amy Hutchinson
5 years ago
Views:

1 Supervised Learning of Complete Morphological Paradigms Greg Durrett and John DeNero UC Berkeley / Google

2 Morphological Inflection train (de) Zug

3 Morphological Inflection train (de) Zug NOM, SING: Zug NOM, PLU: Züge GEN, SING: Zuges GEN, PLU: Züge (8 total)

4 Morphological Inflection train (de) Zug NOM, SING: Zug NOM, PLU: Züge GEN, SING: Zuges GEN, PLU: Züge (8 total) u NOM, SING: u NOM, PLU: ü GEN, SING: u GEN, PLU: ü

5 Morphological Inflection train (de) Zug NOM, SING: Zug NOM, PLU: Züge GEN, SING: Zuges GEN, PLU: Züge (8 total) u NOM, SING: u NOM, PLU: ü GEN, SING: u GEN, PLU: ü g NOM, SING: g NOM, PLU: ge GEN, SING: ges GEN, PLU: ge

6 Morphological Inflection express train (de) Schnellzug NOM, SING: Schnellzug NOM, PLU: Schnellzüge GEN, SING: Schnellzuges GEN, PLU: Schnellzüge u NOM, SING: u NOM, PLU: ü GEN, SING: u GEN, PLU: ü g NOM, SING: g NOM, PLU: ge GEN, SING: ges GEN, PLU: ge

7 Morphological Inflection egg white (fi) valkuainen culprit (fi) syyllinen NOM, SING: valkuainen GEN, SING: valkuaisen PART, SING: valkuaista NOM, SING: syyllinen GEN, SING: syyllisen PART, SING: syyllistä

8 Morphological Inflection egg white (fi) valkuainen culprit (fi) syyllinen NOM, SING: valkuainen GEN, SING: valkuaisen PART, SING: valkuaista NOM, SING: syyllinen GEN, SING: syyllisen PART, SING: syyllistä

9 Morphological Inflection egg white (fi) valkuainen back vowels culprit (fi) syyllinen NOM, SING: valkuainen GEN, SING: valkuaisen PART, SING: valkuaista NOM, SING: syyllinen GEN, SING: syyllisen PART, SING: syyllistä

10 Morphological Inflection egg white (fi) valkuainen back vowels culprit (fi) syyllinen front vowels NOM, SING: valkuainen GEN, SING: valkuaisen PART, SING: valkuaista NOM, SING: syyllinen GEN, SING: syyllisen PART, SING: syyllistä

11 Morphological Inflection egg white (fi) valkuainen back vowels NOM, SING: valkuainen GEN, SING: valkuaisen PART, SING: valkuaista culprit (fi) syyllinen front vowels NOM, SING: syyllinen GEN, SING: syyllisen PART, SING: syyllistä Suffix morphology can depend on context in a complex way

12 Morphological Inflection Given Base form (Word)

13 Morphological Inflection Given Base form (Word) Predict Inflection table (variants) Attributes(1): Inflected form(1) Attributes(2): Inflected form(2)

14 Morphological Inflection Given Base form (Word) Predict Inflection table (variants) Attributes(1): Inflected form(1) Attributes(2): Inflected form(2) Language/POS independence

15 Morphological Inflection Given Base form (Word) Predict Inflection table (variants) Attributes(1): Inflected form(1) Attributes(2): Inflected form(2) Language/POS independence Handle multi-part paradigms

16 Morphological Inflection Given Base form (Word) Predict Inflection table (variants) Attributes(1): Inflected form(1) Attributes(2): Inflected form(2) Language/POS independence Handle multi-part paradigms Rich way to predict inflections

17 Outline

18 Outline Rule extraction (>1000) Zug Kluft

19 Outline Rule extraction (>1000) (<50) Zug Kluft u g

20 Outline Rule extraction (>1000) (<50) Zug Kluft u g ( Paradigm prediction Zug ), u

21 Outline Rule extraction (>1000) (<50) Zug Kluft u g ( Paradigm prediction Zug ), u Base form predictor Inflection table

22 Wiktionary Data Thousands of full inflection tables for many languages and parts of speech

23 Wiktionary Data Thousands of full inflection tables for many languages and parts of speech

24 Wiktionary Data Thousands of full inflection tables for many languages and parts of speech more

25 Previous Work Dreyer and Eisner (2011) develops a semisupervised Bayesian model

26 Previous Work Dreyer and Eisner (2011) develops a semisupervised Bayesian model Wicentowski (2004) predicts one variant at a time (the lemma)

27 Previous Work Dreyer and Eisner (2011) develops a semisupervised Bayesian model Wicentowski (2004) predicts one variant at a time (the lemma) u NOM, SING: u NOM, PLU: ü GEN, SING: u GEN, PLU: ü

28 Rule Extraction

29 Rule Extraction to insist (de) dringen 1P, PRES, SING: dringe 2P, PRES, SING: dringst 3P, PRES, SING: dringt 1P, PAST, SING: drang (27 total)

30 Rule Extraction to insist (de) dr i ngen 1P, PRES, SING: dringe 2P, PRES, SING: dringst 3P, PRES, SING: dringt 1P, PAST, SING: drang (27 total) i 1P, PRES, SING: i 2P, PRES, SING: i 3P, PRES, SING: i 1P, PAST, SING: a

31 Rule Extraction to insist (de) dr i ng en 1P, PRES, SING: dringe 2P, PRES, SING: dringst 3P, PRES, SING: dringt 1P, PAST, SING: drang (27 total) i 1P, PRES, SING: i 2P, PRES, SING: i 3P, PRES, SING: i 1P, PAST, SING: a en 1P, PRES, SING: e 2P, PRES, SING: st 3P, PRES, SING: t 1P, PAST, SING: -

32 Alignment to Base Form d r i n g e 1P, PRES, SING

33 Alignment to Base Form d r i n g e 1P, PRES, SING D

34 Alignment to Base Form 1P, PRES, SING d r i n g e

35 Alignment to Base Form d r i n g s t 2P, PRES, SING 1P, PRES, SING d r i n g e

36 Alignment to Base Form d r i n g s t 2P, PRES, SING S S 1P, PRES, SING d r i n g e

37 Alignment to Base Form 1P, PRES, SING 2P, PRES, SING d r i n g e d r i n g s t

38 Alignment to Base Form d r i n g t 3P, PRES, SING S D 1P, PRES, SING 2P, PRES, SING d r i n g e d r i n g s t

39 Alignment to Base Form 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING d r i n g e d r i n g s t d r i n g t

40 Alignment to Base Form S d r a n g 1P, PAST, SING D D 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING d r i n g e d r i n g s t d r i n g t

41 Alignment to Base Form 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g

42 Concatenation 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g

43 Concatenation 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g

44 Concatenation 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g

45 Concatenation 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g

46 Concatenation 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g

47 Concatenation 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g

48 Concatenation 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g

49 Concatenation 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g

50 Concatenation 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g

51 Concatenation 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g d r i n g e d r i n g s t d r i n g t d r a n g

52 Concatenation 1P, PRES, SING 2P, PRES, SING 3P, PRES, SING 1P, PAST, SING d r i n g e d r i n g s t d r i n g t d r a n g d r i n g e d r i n g s t d r i n g t d r a n g

53 Outline Rule extraction (>1000) (<50) Zug Kluft u g ( Paradigm prediction Zug ), u Base form predictor Inflection table

54 Paradigm Prediction w i n d e n to wind (de)

55 Paradigm Prediction w i n d e n to wind (de) i 1 i i i i a en 1 en e st t -

56 Paradigm Prediction w i n d e n to wind (de) i 1 i i i i a en 1 en e st t - en 2 en e est et - n 1 n - st t te

57 Paradigm Prediction w i n d e n to wind (de) = = = i 1 en 2 i 1 i i i i a en 1 en e st t - en 2 en e est et - n 1 n - st t te

58 Paradigm Prediction w i n d e n to wind (de) = = = = = = i 1 en 2 i 1 i i i i a en 1 en e st t - en 2 en e est et - n 1 n - st t te

59 Paradigm Prediction w i n d e n to wind (de) = = = = = = i 1 n 1 en 2 i 1 i i i i a en 1 en e st t - en 2 en e est et - n 1 n - st t te

60 Paradigm Prediction w i n d e n to wind (de) = = = = = = i 1 n 1 n 1 en 1 en 2 i 1 i i i i a en 1 en e st t - en 2 en e est et - n 1 n - st t te

61 Paradigm Prediction w i n d e n = = = = = = i 1 n 1 n 1 en 1 en 2

62 Paradigm Prediction w i n d e n = = = = = = i 1 n 1 n 1 en 1 en 2 Paths through this lattice are hypotheses

63 Paradigm Prediction w i n d e n = = = = = = i 1 n 1 n 1 en 1 en 2 Paths through this lattice are hypotheses Model with semi-markov CRF (Sarawagi and Cohen 2004)

64 Prediction Features w i n d e n i 1

65 Prediction Features w i n d e n i 1 binden verbinden

66 Prediction Features w i n d e n i 1 binden verbinden

67 Prediction Features w i n d e n i 1 [i1] nd binden verbinden

68 Prediction Features w i n d e n i 1 [i1] nd binden verbinden Rule identity conjoined with 1- through 4-grams at offsets up to +/-5

69 Prediction Features w i n d e n i 1 [i1] nd binden verbinden Rule identity conjoined with 1- through 4-grams at offsets up to +/-5 Coarse features shared between different rules

70 Learning The gold inflection table of every training example can be produced using our rules

71 Learning The gold inflection table of every training example can be produced using our rules Optimize conditional log-likelihood of the correct rule sequence

72 Outline Rule extraction (>1000) (<50) Zug Kluft u g ( Paradigm prediction Zug ), u Base form predictor Inflection table

73 Wiktionary Evaluation Setup # lines/table Train size Test size German nouns German verbs Spanish verbs Finnish nouns/adjs Finnish verbs

74 Wiktionary Evaluation Setup # lines/table Train size Test size German nouns German verbs Spanish verbs Finnish nouns/adjs Finnish verbs Most common 200 inflection tables are not chosen for the test set; we can memorize these

75 Wiktionary Results NAÏVE 100 THIS WORK ORACLE FACTORED 75 Average inflected form accuracy Suffixing baseline

76 Wiktionary Results NAÏVE 100 THIS WORK ORACLE FACTORED 75 Average inflected form accuracy

77 Wiktionary Results NAÏVE 100 THIS WORK ORACLE FACTORED 75 Average inflected form accuracy The best we can do with our rules

78 Wiktionary Results 100 NAÏVE THIS WORK ORACLE FACTORED 75 Average inflected form accuracy Separate predictor for each inflected form

79 Wiktionary Results NAÏVE 100 THIS WORK ORACLE FACTORED 75 Average whole table accuracy

80 Dreyer and Eisner (2011)

81 Dreyer and Eisner (2011) Evaluate on German verbs in CELEX Results averaged over 10 random train/test splits Small train sets (50 or 100 observed tables) Large test sets (5415 verbs)

82 Dreyer and Eisner (2011) Evaluate on German verbs in CELEX Results averaged over 10 random train/test splits Small train sets (50 or 100 observed tables) Large test sets (5415 verbs) Hierarchical Bayesian model of inflection Type-level transducers of variants can be trained in a supervised fashion Additionally incorporate unlabeled text with a token-level model

83 CELEX Results Inflected form accuracy DE11 DE11+CORPUS THIS WORK Training set size (example tables)

84 CELEX Results Inflected form accuracy DE11 DE11+CORPUS THIS WORK Training set size (example tables)

85 Conclusion Morphological inflection rules can be learned from supervised data, which is widely available

86 Conclusion Morphological inflection rules can be learned from supervised data, which is widely available Structured prediction of entire tables at once is effective for inflecting unseen base forms

87 Conclusion Morphological inflection rules can be learned from supervised data, which is widely available Structured prediction of entire tables at once is effective for inflecting unseen base forms Code and Wiktionary data is available at

88 Conclusion Morphological inflection rules can be learned from supervised data, which is widely available Structured prediction of entire tables at once is effective for inflecting unseen base forms Code and Wiktionary data is available at Thank you!

89 Accuracy Breakdown Inflected form accuracy THIS WORK DE verbs DE nouns ES verbs FI verbs FI nouns/adjs

Sarcasm Detection in Text: Design Document

CSC 59866 Senior Design Project Specification Professor Jie Wei Wednesday, November 23, 2016 Sarcasm Detection in Text: Design Document Jesse Feinman, James Kasakyan, Jeff Stolzenberg 1 Table of contents