Detecting Attempts at Humor in Multiparty Meetings
|
|
|
- Blanche Casey
- 5 years ago
- Views:
Transcription
1 Detecting Attempts at Humor in Multiparty Meetings Kornel Laskowski Carnegie Mellon University Pittsburgh PA, USA 14 September, 2008 K. Laskowski ICSC 2009, Berkeley CA, USA 1/26
2 Why bother with humor? generally, systems assume uniform truth across utterances humans do not make that assumption a speaker may be unconcerned how their utterance is interpreted but a speaker may covertly perform extra work to pass off as true/serious that which is not speaker is not helping us detect their effort (e.g. lying) or a speaker may overtly perform extra work to pass off as untrue/unserious that which may be taken at face value speaker is helping us detect their effort (e.g. joking) need to detect grades of truth, at least when speakers are collaborative K. Laskowski ICSC 2009, Berkeley CA, USA 2/26
3 Why bother with humor (part II)? humor plays a socially cohesive role creates vehicle for expressing, maintaining, constructing, dissolving interpersonal relationships systems must detect it, or miss important important cues underlying variability across participants to conversation K. Laskowski ICSC 2009, Berkeley CA, USA 3/26
4 Why bother with humor (part III)? humor does not occur uniformly in time its occurrence is colocated with segment boundaries at the detection may be helpful to segmentation of conversation at the turn level topic level meta-conversation level systems must detect it, or miss important cues underlying variability across time in conversation K. Laskowski ICSC 2009, Berkeley CA, USA 4/26
5 Outline of this Talk 1 Introduction 2 Humor in our Data 3 HMM Decoder Framework baseline (oracle) lexical features 4 Modeling Conversational Context speech activity/interaction features laughter activity/interaction features 5 Analysis 6 Conclusions & Recommendations K. Laskowski ICSC 2009, Berkeley CA, USA 5/26
6 Potential Impact of Modeling Laughter must determine if current speaker is intending to amuse task may be too hard for a computer instead, let humans do the work offline: wait to see if others laugh even if attempt to amuse fails, others may laugh to show that they understand the utterance is not meant seriously online: wait to see if speaker laughs to show that utterance is not meant seriously SPKR A JOKE SPKR B SPKR C K. Laskowski t t ICSC , Berkeley t + 2 CA, USA 6/26
7 Potential Impact of Modeling Laughter must determine if current speaker is intending to amuse task may be too hard for a computer instead, let humans do the work offline: wait to see if others laugh even if attempt to amuse fails, others may laugh to show that they understand the utterance is not meant seriously online: wait to see if speaker laughs to show that utterance is not meant seriously SPKR A JOKE SPKR B LAUGH SPKR C LAUGH K. Laskowski t t ICSC , Berkeley t + 2 CA, USA 6/26
8 Potential Impact of Modeling Laughter must determine if current speaker is intending to amuse task may be too hard for a computer instead, let humans do the work offline: wait to see if others laugh even if attempt to amuse fails, others may laugh to show that they understand the utterance is not meant seriously online: wait to see if speaker laughs to show that utterance is not meant seriously SPKR A JOKE LAUGH SPKR B LAUGH SPKR C LAUGH K. Laskowski t t ICSC , Berkeley t + 2 CA, USA 6/26
9 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE Laskowski & Burger, 2006 Neiberg et al, 2006 EMOT. INVOLVED SPEECH Wrede & Shriberg, 2003 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
10 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE Laskowski & Burger, 2006 Neiberg et al, 2006 EMOT. INVOLVED SPEECH Wrede & Shriberg, 2003 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
11 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE Laskowski & Burger, 2006 Neiberg et al, 2006 EMOT. INVOLVED SPEECH Wrede & Shriberg, 2003 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
12 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE Laskowski & Burger, 2006 Neiberg et al, 2006 EMOT. INVOLVED SPEECH Wrede & Shriberg, 2003 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
13 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE Laskowski & Burger, 2006 Neiberg et al, 2006 EMOT. INVOLVED SPEECH Wrede & Shriberg, 2003 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
14 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE EMOT. INVOLVED SPEECH Laskowski & Burger, 2006 Wrede & Shriberg, 2003 Neiberg et al, 2006 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
15 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE Laskowski & Burger, 2006 Neiberg et al, 2006 EMOT. INVOLVED SPEECH Wrede & Shriberg, 2003 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
16 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE Laskowski & Burger, 2006 Neiberg et al, 2006 EMOT. INVOLVED SPEECH Wrede & Shriberg, 2003 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
17 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE Laskowski & Burger, 2006 Neiberg et al, 2006 EMOT. INVOLVED SPEECH Wrede & Shriberg, 2003 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
18 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE Laskowski & Burger, 2006 Neiberg et al, 2006 EMOT. INVOLVED SPEECH Wrede & Shriberg, 2003 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
19 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE EMOT. INVOLVED SPEECH Laskowski & Burger, 2006 Wrede & Shriberg, 2003 Neiberg et al, 2006 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
20 Computational Context and Prior Work SENTIMENT Somasundaran et al, 2007 HUMOR Clark & Popescu Belis, 2004 EMOTIONAL VALENCE EMOT. INVOLVED SPEECH Laskowski & Burger, 2006 Wrede & Shriberg, 2003 Neiberg et al, 2006 Laskowski, 2008 SPEECH RECOGNITION SPEECH ACTIVITY PROSODIC MODELING LAUGHTER ACTIVITY Kennedy & Ellis, 2004 Truong & van Leeuwen, 2005 Knox & Mirghafori, 2007 AUDIO K. Laskowski ICSC 2009, Berkeley CA, USA 7/26
21 ICSI Meeting Corpus (Janin et al, 2003; Shriberg et al, 2004) naturally occurring meetings 75 meetings, 66 hours of meeting time TrainSet: 51 meetings DevSet: 11 meetings EvalSet: 11 meetings 3-9 participants per meeting different types unstructured discussion among peers round-table reporting among peers 1 professor and N students meetings human-transcribed words (with forced-alignment), dialog acts K. Laskowski ICSC 2009, Berkeley CA, USA 8/26
22 Humor Annotation in ICSI Meetings Based on the 8 DA types studied in Laskowski & Shriberg, Modeling Other Talkers for Improved Dialog Act Recognition in Meetings, INTERSPEECH Propositional Content DA Types statement question s q 85% 6.6% Feedback DA Types backchannel acknowledgment assert Floor Mechanism DA Types b 2.8% floor holder fh 2.5% bk 1.4% floor grabber fg 0.6% aa 1.1% hold h 0.3% K. Laskowski ICSC 2009, Berkeley CA, USA 9/26
23 Humor Annotation in ICSI Meetings Based on the 8 DA types studied in Laskowski & Shriberg, Modeling Other Talkers for Improved Dialog Act Recognition in Meetings, INTERSPEECH Propositional Content DA Types statement question s q 85% 6.6% joke Humor-Bearing DA Types j 0.6% Feedback DA Types backchannel acknowledgment assert Floor Mechanism DA Types b 2.8% floor holder fh 2.5% bk 1.4% floor grabber fg 0.6% aa 1.1% hold h 0.3% K. Laskowski ICSC 2009, Berkeley CA, USA 9/26
24 Goal of this Work SPKR A: SPKR B: SPKR C: SPKR D: K. Laskowski ICSC 2009, Berkeley CA, USA 10/26
25 Goal of this Work SPKR A: SPKR B: SPKR C: SPKR D: TALKSPURT K. Laskowski ICSC 2009, Berkeley CA, USA 10/26
26 Goal of this Work SPKR A: SPKR B: SPKR C: SPKR D: LAUGHBOUT K. Laskowski ICSC 2009, Berkeley CA, USA 10/26
27 Goal of this Work SPKR A: SPKR B: SPKR C: SPKR D: TASK: find speech which is humor-bearing K. Laskowski ICSC 2009, Berkeley CA, USA 10/26
28 Goal of this Work SPKR A: SPKR B: SPKR C: SPKR D: TASK: find speech which is humor-bearing (DA segmentation and recognition, with focus on a subset of DAs) K. Laskowski ICSC 2009, Berkeley CA, USA 10/26
29 Talkspurt (TS) Boundaries DA Boundaries SPKR A: SPKR B: SPKR C: SPKR D: decoding the state of one participant at a time may have 1:1 correspondence between DAs and TSs and 1:1 correspondence between DA-gaps and TS-gaps but may also have TS gaps inside DAs 1:N correspondence between DAs and TSs explicitly model intra-da silence opposite (N:1 correspondence) may also occur entertain possibility that DA boundaries occur anywhere K. Laskowski ICSC 2009, Berkeley CA, USA 11/26
30 Talkspurt (TS) Boundaries DA Boundaries SPKR B: decoding the state of one participant at a time may have 1:1 correspondence between DAs and TSs and 1:1 correspondence between DA-gaps and TS-gaps but may also have TS gaps inside DAs 1:N correspondence between DAs and TSs explicitly model intra-da silence opposite (N:1 correspondence) may also occur entertain possibility that DA boundaries occur anywhere K. Laskowski ICSC 2009, Berkeley CA, USA 11/26
31 Talkspurt (TS) Boundaries DA Boundaries SPKR B: decoding the state of one participant at a time may have 1:1 correspondence between DAs and TSs and 1:1 correspondence between DA-gaps and TS-gaps but may also have TS gaps inside DAs 1:N correspondence between DAs and TSs explicitly model intra-da silence opposite (N:1 correspondence) may also occur entertain possibility that DA boundaries occur anywhere K. Laskowski ICSC 2009, Berkeley CA, USA 11/26
32 Talkspurt (TS) Boundaries DA Boundaries SPKR B: TALKSPURT DIALOG ACT decoding the state of one participant at a time may have 1:1 correspondence between DAs and TSs and 1:1 correspondence between DA-gaps and TS-gaps but may also have TS gaps inside DAs 1:N correspondence between DAs and TSs explicitly model intra-da silence opposite (N:1 correspondence) may also occur entertain possibility that DA boundaries occur anywhere K. Laskowski ICSC 2009, Berkeley CA, USA 11/26
33 Talkspurt (TS) Boundaries DA Boundaries SPKR B: TALKSPURT DIALOG ACT decoding the state of one participant at a time may have 1:1 correspondence between DAs and TSs and 1:1 correspondence between DA-gaps and TS-gaps but may also have TS gaps inside DAs 1:N correspondence between DAs and TSs explicitly model intra-da silence opposite (N:1 correspondence) may also occur entertain possibility that DA boundaries occur anywhere K. Laskowski ICSC 2009, Berkeley CA, USA 11/26
34 Talkspurt (TS) Boundaries DA Boundaries SPKR B: TALKSPURT DIALOG ACT decoding the state of one participant at a time may have 1:1 correspondence between DAs and TSs and 1:1 correspondence between DA-gaps and TS-gaps but may also have TS gaps inside DAs 1:N correspondence between DAs and TSs explicitly model intra-da silence opposite (N:1 correspondence) may also occur entertain possibility that DA boundaries occur anywhere K. Laskowski ICSC 2009, Berkeley CA, USA 11/26
35 Talkspurt (TS) Boundaries DA Boundaries SPKR B: TALKSPURT DIALOG ACT decoding the state of one participant at a time may have 1:1 correspondence between DAs and TSs and 1:1 correspondence between DA-gaps and TS-gaps but may also have TS gaps inside DAs 1:N correspondence between DAs and TSs explicitly model intra-da silence opposite (N:1 correspondence) may also occur entertain possibility that DA boundaries occur anywhere K. Laskowski ICSC 2009, Berkeley CA, USA 11/26
36 Talkspurt (TS) Boundaries DA Boundaries SPKR B: TALKSPURT DIALOG ACT decoding the state of one participant at a time may have 1:1 correspondence between DAs and TSs and 1:1 correspondence between DA-gaps and TS-gaps but may also have TS gaps inside DAs 1:N correspondence between DAs and TSs explicitly model intra-da silence opposite (N:1 correspondence) may also occur entertain possibility that DA boundaries occur anywhere K. Laskowski ICSC 2009, Berkeley CA, USA 11/26
37 Talkspurt (TS) Boundaries DA Boundaries SPKR B: TALKSPURT DIALOG ACT decoding the state of one participant at a time may have 1:1 correspondence between DAs and TSs and 1:1 correspondence between DA-gaps and TS-gaps but may also have TS gaps inside DAs 1:N correspondence between DAs and TSs explicitly model intra-da silence opposite (N:1 correspondence) may also occur entertain possibility that DA boundaries occur anywhere K. Laskowski ICSC 2009, Berkeley CA, USA 11/26
38 Talkspurt (TS) Boundaries DA Boundaries SPKR B: TALKSPURT DIALOG ACT decoding the state of one participant at a time may have 1:1 correspondence between DAs and TSs and 1:1 correspondence between DA-gaps and TS-gaps but may also have TS gaps inside DAs 1:N correspondence between DAs and TSs explicitly model intra-da silence opposite (N:1 correspondence) may also occur entertain possibility that DA boundaries occur anywhere K. Laskowski ICSC 2009, Berkeley CA, USA 11/26
39 Talkspurt (TS) Boundaries DA Boundaries SPKR B: TALKSPURT DIALOG ACT decoding the state of one participant at a time may have 1:1 correspondence between DAs and TSs and 1:1 correspondence between DA-gaps and TS-gaps but may also have TS gaps inside DAs 1:N correspondence between DAs and TSs explicitly model intra-da silence opposite (N:1 correspondence) may also occur entertain possibility that DA boundaries occur anywhere K. Laskowski ICSC 2009, Berkeley CA, USA 11/26
40 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
41 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
42 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
43 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
44 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
45 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
46 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
47 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
48 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
49 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
50 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
51 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
52 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
53 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
54 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
55 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
56 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
57 Proposed HMM Sub-Topology for DAs NON DA TERMINAL TALKSPURT FRAGMENT INTRA DA TALKSPURT GAP DA TERMINAL TALKSPURT FRAGMENT ENTRY EGRESS SPKR B: K. Laskowski ICSC 2009, Berkeley CA, USA 12/26
58 Proposed HMM Topology for Conversational Speech the complete topology consists of a DA sub-topology for each of 9 DA types fully connected via inter-da GAP subnetworks s j aa q b h fh fg bk K. Laskowski ICSC 2009, Berkeley CA, USA 13/26
59 Oracle Lexical Features each 100 ms frame of speech can be assigned to one word w assign to that frame the emission probability: of the bigram of which w is the right token, and of the bigram of wihch w is the left token train a generative model over left and right bigrams for each HMM state bigrams whose probability of occurrence for any DA type is < 0.1% are mapped to UNK K. Laskowski ICSC 2009, Berkeley CA, USA 14/26
60 Baseline Performance w/o T fully-connected topology, equiprobable transitions w/ T0 proposed topology, equiprobable transitions w/ T1 proposed topology, transitions trained using TrainSet (ML) System DevSet EvalSet FA MS ERR FA MS ERR T T LEX w/o T LEX w/ T LEX w/ T K. Laskowski ICSC 2009, Berkeley CA, USA 15/26
61 Speech Activity/Interaction Features, S OTH1: SPKR: OTH2: OTH3: OTH4: decoding one participant (SPKR) at a time at instant t, model the thumbnail image of context consider a temporal context of width T want invariance under participant-index rotation rank OTH participants by local speaking time want a fixed-size feature vector: consider only K others model features using state-specific GMMs (after LDA) K. Laskowski ICSC 2009, Berkeley CA, USA 16/26
62 Speech Activity/Interaction Features, S OTH1: SPKR: OTH2: OTH3: OTH4: decoding one participant (SPKR) at a time at instant t, model the thumbnail image of context consider a temporal context of width T want invariance under participant-index rotation rank OTH participants by local speaking time want a fixed-size feature vector: consider only K others model features using state-specific GMMs (after LDA) K. Laskowski ICSC 2009, Berkeley CA, USA 16/26
63 Speech Activity/Interaction Features, S OTH1: SPKR: OTH2: OTH3: OTH4: T/2 T/2 decoding one participant (SPKR) at a time at instant t, model the thumbnail image of context consider a temporal context of width T want invariance under participant-index rotation rank OTH participants by local speaking time want a fixed-size feature vector: consider only K others model features using state-specific GMMs (after LDA) K. Laskowski ICSC 2009, Berkeley CA, USA 16/26
64 Speech Activity/Interaction Features, S OTH1: SPKR: OTH2: OTH3: OTH4: T/2 T/2 decoding one participant (SPKR) at a time at instant t, model the thumbnail image of context consider a temporal context of width T want invariance under participant-index rotation rank OTH participants by local speaking time want a fixed-size feature vector: consider only K others model features using state-specific GMMs (after LDA) K. Laskowski ICSC 2009, Berkeley CA, USA 16/26
65 Speech Activity/Interaction Features, S OTH1: SPKR: OTH4: OTH3: OTH2: T/2 T/2 decoding one participant (SPKR) at a time at instant t, model the thumbnail image of context consider a temporal context of width T want invariance under participant-index rotation rank OTH participants by local speaking time want a fixed-size feature vector: consider only K others model features using state-specific GMMs (after LDA) K. Laskowski ICSC 2009, Berkeley CA, USA 16/26
66 Speech Activity/Interaction Features, S SPKR: OTH1: OTH2: OTH3: OTH4: T/2 T/2 decoding one participant (SPKR) at a time at instant t, model the thumbnail image of context consider a temporal context of width T want invariance under participant-index rotation rank OTH participants by local speaking time want a fixed-size feature vector: consider only K others model features using state-specific GMMs (after LDA) K. Laskowski ICSC 2009, Berkeley CA, USA 16/26
67 Speech Activity/Interaction Features, S SPKR: OTH1: OTH2: OTH3: OTH4: K T/2 T/2 decoding one participant (SPKR) at a time at instant t, model the thumbnail image of context consider a temporal context of width T want invariance under participant-index rotation rank OTH participants by local speaking time want a fixed-size feature vector: consider only K others model features using state-specific GMMs (after LDA) K. Laskowski ICSC 2009, Berkeley CA, USA 16/26
68 Speech Activity/Interaction Features, S SPKR: OTH1: OTH2: OTH3: OTH4: K FEATURE "VECTOR" T/2 T/2 decoding one participant (SPKR) at a time at instant t, model the thumbnail image of context consider a temporal context of width T want invariance under participant-index rotation rank OTH participants by local speaking time want a fixed-size feature vector: consider only K others model features using state-specific GMMs (after LDA) K. Laskowski ICSC 2009, Berkeley CA, USA 16/26
69 Speech Activity/Interaction Features, S SPKR: OTH1: OTH2: OTH3: OTH4: K FEATURE "VECTOR" T/2 T/2 decoding one participant (SPKR) at a time at instant t, model the thumbnail image of context consider a temporal context of width T want invariance under participant-index rotation rank OTH participants by local speaking time want a fixed-size feature vector: consider only K others model features using state-specific GMMs (after LDA) K. Laskowski ICSC 2009, Berkeley CA, USA 16/26
70 Laughter Activity/Interaction Features, L process same as for speech activity/interaction features: 1 sort others by amount of laughing time in T-width window 2 extract features from K most-laughing others may be suboptimal (too complex overfit) laughter accounts for 9.6% of vocalizing time in the paper, also consider subsetting all laughter bouts into: voiced bouts (approx. 2 /3 of laughter by time) unvoiced bouts (approx. 1 /3 of laughter by time) K. Laskowski ICSC 2009, Berkeley CA, USA 17/26
71 System Combination 1 model-space combination ( M ) P ([F S,F L ] [M S, M L ]) P (F S M S ) P (F L M L ) F S F L = f (K,rank (S),S) = f (K,rank (L),L) 2 feature-space combination ( F ) P ([F S,F L ] [M S, M L ]) P ([F S,F L ] M S L ) F S F L = f (K,rank (S),S) = f (K,rank (L),L) 3 feature-computation-space combination ( C ) P ([F S,F L ] [M S, M L ]) P ([F S,F L ] M S L ) F S F L = f (K,rank (S L),S) = f (K,rank (S L),L) K. Laskowski ICSC 2009, Berkeley CA, USA 18/26
72 Results System DevSet EvalSet FA MS ERR FA MS ERR LEX S L S M L S F L S C L LEX M S M L L is the best single source of information for this task model-space combination with S leads to improvement combination with LEX leads to improvement on DevSet only K. Laskowski ICSC 2009, Berkeley CA, USA 19/26
73 Receiver Operating Characteristics (DevSet) 100 TRUE POSITIVE RATE (%) LEX S L LEX+S+L no discr. equal error FALSE POSITIVE RATE (%) K. Laskowski ICSC 2009, Berkeley CA, USA 20/26
74 Interpreting Emission Probability Diagrams condition: given an event of type A occurring at time t what is the likelihood that an event of type B occurs at time t [t 5,t + 5] retrain single-gaussian model on unnormalized features PROBABILITY OF OCCURRENCE OF B TIME OF OCCURRENCE OF B K. Laskowski ICSC 2009, Berkeley CA, USA 21/26
75 Interlocutor Laughter Context at DA Termination j DAs j DAs locally 2nd most laughing locally 1st most laughing K. Laskowski ICSC 2009, Berkeley CA, USA 22/26
76 Interlocutor Laughter Context at DA Termination j DAs j DAs locally 2nd most laughing locally 1st most laughing K. Laskowski ICSC 2009, Berkeley CA, USA 22/26
77 Interlocutor Laughter Context at DA Termination j DAs j DAs locally 2nd most laughing locally 1st most laughing K. Laskowski ICSC 2009, Berkeley CA, USA 22/26
78 Interlocutor Laughter Context at DA Termination j DAs j DAs locally 2nd most laughing locally 1st most laughing K. Laskowski ICSC 2009, Berkeley CA, USA 22/26
79 Interlocutor Laughter Context at DA Termination j DAs j DAs locally 2nd most laughing locally 1st most laughing K. Laskowski ICSC 2009, Berkeley CA, USA 22/26
80 Interlocutor Laughter Context at DA Termination j DAs j DAs locally 2nd most laughing locally 1st most laughing K. Laskowski ICSC 2009, Berkeley CA, USA 22/26
81 Target Speaker Laughter Context j DAs j DAs target speaker How well we do with laughter only from the target speaker? K. Laskowski ICSC 2009, Berkeley CA, USA 23/26
82 Target Speaker Laughter Context j DAs j DAs target speaker How well we do with laughter only from the target speaker? K. Laskowski ICSC 2009, Berkeley CA, USA 23/26
83 Target Speaker Laughter Context j DAs j DAs target speaker How well we do with laughter only from the target speaker? K. Laskowski ICSC 2009, Berkeley CA, USA 23/26
84 Target Speaker Laughter Context j DAs j DAs target speaker How well we do with laughter only from the target speaker? K. Laskowski ICSC 2009, Berkeley CA, USA 23/26
85 Target Speaker Laughter Context j DAs j DAs target speaker How well we do with laughter only from the target speaker? K. Laskowski ICSC 2009, Berkeley CA, USA 23/26
86 Target Speaker Laughter Context j DAs j DAs target speaker How well we do with laughter only from the target speaker? System DevSet EvalSet FA MS ERR FA MS ERR S L L K. Laskowski ICSC 2009, Berkeley CA, USA 23/26
87 Interlocutor j-speech Context at j-da Termination target speaker locally 1st most j-talkative interlocutor locally 2nd most j-talkative interlocutor K. Laskowski ICSC 2009, Berkeley CA, USA 24/26
88 Interlocutor j-speech Context at j-da Termination target speaker locally 1st most j-talkative interlocutor locally 2nd most j-talkative interlocutor K. Laskowski ICSC 2009, Berkeley CA, USA 24/26
89 Interlocutor j-speech Context at j-da Termination target speaker locally 1st most j-talkative interlocutor locally 2nd most j-talkative interlocutor K. Laskowski ICSC 2009, Berkeley CA, USA 24/26
90 Summary GOAL: detect humor-bearing speech APPROACH: frame-level HMM decoding consider multiparticipant speech & laughter context RESULTS: 1 at FPRs of 5% (DevSet): lexical features yield TPRs 4 higher than random guessing speech context yields TPRs 2 higher than lexical features laughter context yields TPRs 2 higher than speech context 2 laughter context features: EER < 24% (EvalSet) 3 model-space combination improves EERs by 5% abs 4 locally most laughing interlocutor more likely to laugh than not 5 evidence that jokers themselves laugh, perhaps to signal intent 6 at most 2 participants likely to joke in any 10 second interval K. Laskowski ICSC 2009, Berkeley CA, USA 25/26
91 THANK YOU Special thanks to Liz Shriberg, for: access to the ICSI MRDA annotations helpful discussion during this work K. Laskowski ICSC 2009, Berkeley CA, USA 26/26
Analysis of the Occurrence of Laughter in Meetings
Analysis of the Occurrence of Laughter in Meetings Kornel Laskowski 1,2 & Susanne Burger 2 1 interact, Universität Karlsruhe 2 interact, Carnegie Mellon University August 29, 2007 Introduction primary
Automatic Laughter Detection
Automatic Laughter Detection Mary Knox Final Project (EECS 94) knoxm@eecs.berkeley.edu December 1, 006 1 Introduction Laughter is a powerful cue in communication. It communicates to listeners the emotional
Improving Frame Based Automatic Laughter Detection
Improving Frame Based Automatic Laughter Detection Mary Knox EE225D Class Project knoxm@eecs.berkeley.edu December 13, 2007 Abstract Laughter recognition is an underexplored area of research. My goal for
Automatic Laughter Segmentation. Mary Tai Knox
Automatic Laughter Segmentation Mary Tai Knox May 22, 2008 Abstract Our goal in this work was to develop an accurate method to identify laughter segments, ultimately for the purpose of speaker recognition.
Automatic Laughter Detection
Automatic Laughter Detection Mary Knox 1803707 knoxm@eecs.berkeley.edu December 1, 006 Abstract We built a system to automatically detect laughter from acoustic features of audio. To implement the system,
Narrative Theme Navigation for Sitcoms Supported by Fan-generated Scripts
Narrative Theme Navigation for Sitcoms Supported by Fan-generated Scripts Gerald Friedland, Luke Gottlieb, Adam Janin International Computer Science Institute (ICSI) Presented by: Katya Gonina What? Novel
Automatic discrimination between laughter and speech
Speech Communication 49 (2007) 144 158 www.elsevier.com/locate/specom Automatic discrimination between laughter and speech Khiet P. Truong *, David A. van Leeuwen TNO Human Factors, Department of Human
First Step Towards Enhancing Word Embeddings with Pitch Accents for DNN-based Slot Filling on Recognized Text
First Step Towards Enhancing Word Embeddings with Pitch Accents for DNN-based Slot Filling on Recognized Text Sabrina Stehwien, Ngoc Thang Vu IMS, University of Stuttgart March 16, 2017 Slot Filling sequential
A Phonetic Analysis of Natural Laughter, for Use in Automatic Laughter Processing Systems
A Phonetic Analysis of Natural Laughter, for Use in Automatic Laughter Processing Systems Jérôme Urbain and Thierry Dutoit Université de Mons - UMONS, Faculté Polytechnique de Mons, TCTS Lab 20 Place du
Narrative Theme Navigation for Sitcoms Supported by Fan-generated Scripts
Narrative Theme Navigation for Sitcoms Supported by Fan-generated Scripts Gerald Friedland International Computer Science Institute 1947 Center Street, Suite 600 Berkeley, CA 94704-1198 fractor@icsi.berkeley.edu
Browsing News and Talk Video on a Consumer Electronics Platform Using Face Detection
Browsing News and Talk Video on a Consumer Electronics Platform Using Face Detection Kadir A. Peker, Ajay Divakaran, Tom Lanning Mitsubishi Electric Research Laboratories, Cambridge, MA, USA {peker,ajayd,}@merl.com
Phone-based Plosive Detection
Phone-based Plosive Detection 1 Andreas Madsack, Grzegorz Dogil, Stefan Uhlich, Yugu Zeng and Bin Yang Abstract We compare two segmentation approaches to plosive detection: One aproach is using a uniform
Formalizing Irony with Doxastic Logic
Formalizing Irony with Doxastic Logic WANG ZHONGQUAN National University of Singapore April 22, 2015 1 Introduction Verbal irony is a fundamental rhetoric device in human communication. It is often characterized
Smile and Laughter in Human-Machine Interaction: a study of engagement
Smile and ter in Human-Machine Interaction: a study of engagement Mariette Soury 1,2, Laurence Devillers 1,3 1 LIMSI-CNRS, BP133, 91403 Orsay cedex, France 2 University Paris 11, 91400 Orsay, France 3
Story Tracking in Video News Broadcasts. Ph.D. Dissertation Jedrzej Miadowicz June 4, 2004
Story Tracking in Video News Broadcasts Ph.D. Dissertation Jedrzej Miadowicz June 4, 2004 Acknowledgements Motivation Modern world is awash in information Coming from multiple sources Around the clock
Analysis and modeling of the role of laughter in Motivational Interviewing based psychotherapy conversations
INTERSPEECH 215 Analysis and modeling of the role of laughter in Motivational Interviewing based psychotherapy conversations Rahul Gupta 1, Theodora Chaspari 1, Panayiotis Georgiou 1, David Atkins 2, Shrikanth
Transcription of the Singing Melody in Polyphonic Music
Transcription of the Singing Melody in Polyphonic Music Matti Ryynänen and Anssi Klapuri Institute of Signal Processing, Tampere University Of Technology P.O.Box 553, FI-33101 Tampere, Finland {matti.ryynanen,
Humor: Prosody Analysis and Automatic Recognition for F * R * I * E * N * D * S *
Humor: Prosody Analysis and Automatic Recognition for F * R * I * E * N * D * S * Amruta Purandare and Diane Litman Intelligent Systems Program University of Pittsburgh amruta,litman @cs.pitt.edu Abstract
Comparison Parameters and Speaker Similarity Coincidence Criteria:
Comparison Parameters and Speaker Similarity Coincidence Criteria: The Easy Voice system uses two interrelating parameters of comparison (first and second error types). False Rejection, FR is a probability
Automatic Labelling of tabla signals
ISMIR 2003 Oct. 27th 30th 2003 Baltimore (USA) Automatic Labelling of tabla signals Olivier K. GILLET, Gaël RICHARD Introduction Exponential growth of available digital information need for Indexing and
... A Pseudo-Statistical Approach to Commercial Boundary Detection. Prasanna V Rangarajan Dept of Electrical Engineering Columbia University
A Pseudo-Statistical Approach to Commercial Boundary Detection........ Prasanna V Rangarajan Dept of Electrical Engineering Columbia University pvr2001@columbia.edu 1. Introduction Searching and browsing
Fusion for Audio-Visual Laughter Detection
Fusion for Audio-Visual Laughter Detection Boris Reuderink September 13, 7 2 Abstract Laughter is a highly variable signal, and can express a spectrum of emotions. This makes the automatic detection of
On Human Capability and Acoustic Cues for Discriminating Singing and Speaking Voices
On Human Capability and Acoustic Cues for Discriminating Singing and Speaking Voices Yasunori Ohishi 1 Masataka Goto 3 Katunobu Itou 2 Kazuya Takeda 1 1 Graduate School of Information Science, Nagoya University,
Laughter and Topic Transition in Multiparty Conversation
Laughter and Topic Transition in Multiparty Conversation Emer Gilmartin, Francesca Bonin, Carl Vogel, Nick Campbell Trinity College Dublin {gilmare, boninf, vogel, nick}@tcd.ie Abstract This study explores
A System for Automatic Chord Transcription from Audio Using Genre-Specific Hidden Markov Models
A System for Automatic Chord Transcription from Audio Using Genre-Specific Hidden Markov Models Kyogu Lee Center for Computer Research in Music and Acoustics Stanford University, Stanford CA 94305, USA
Laughbot: Detecting Humor in Spoken Language with Language and Audio Cues
Laughbot: Detecting Humor in Spoken Language with Language and Audio Cues Kate Park katepark@stanford.edu Annie Hu anniehu@stanford.edu Natalie Muenster ncm000@stanford.edu Abstract We propose detecting
AUTOMATIC RECOGNITION OF LAUGHTER
AUTOMATIC RECOGNITION OF LAUGHTER USING VERBAL AND NON-VERBAL ACOUSTIC FEATURES Tomasz Jacykiewicz 1 Dr. Fabien Ringeval 2 JANUARY, 2014 DEPARTMENT OF INFORMATICS - MASTER PROJECT REPORT Département d
Computational Modelling of Harmony
Computational Modelling of Harmony Simon Dixon Centre for Digital Music, Queen Mary University of London, Mile End Rd, London E1 4NS, UK simon.dixon@elec.qmul.ac.uk http://www.elec.qmul.ac.uk/people/simond
LAUGHTER serves as an expressive social signal in human
Audio-Facial Laughter Detection in Naturalistic Dyadic Conversations Bekir Berker Turker, Yucel Yemez, Metin Sezgin, Engin Erzin 1 Abstract We address the problem of continuous laughter detection over
Research Article Drum Sound Detection in Polyphonic Music with Hidden Markov Models
Hindawi Publishing Corporation EURASIP Journal on Audio, Speech, and Music Processing Volume 2009, Article ID 497292, 9 pages doi:10.1155/2009/497292 Research Article Drum Sound Detection in Polyphonic
Laughter and Smile Processing for Human-Computer Interactions
Laughter and Smile Processing for Human-Computer Interactions Kevin El Haddad, Hüseyin Çakmak, Stéphane Dupont, Thierry Dutoit TCTS lab - University of Mons 31 Boulevard Dolez, 7000, Mons Belgium kevin.elhaddad@umons.ac.be
An Impact Analysis of Features in a Classification Approach to Irony Detection in Product Reviews
Universität Bielefeld June 27, 2014 An Impact Analysis of Features in a Classification Approach to Irony Detection in Product Reviews Konstantin Buschmeier, Philipp Cimiano, Roman Klinger Semantic Computing
Analysis of Visual Similarity in News Videos with Robust and Memory-Efficient Image Retrieval
Analysis of Visual Similarity in News Videos with Robust and Memory-Efficient Image Retrieval David Chen, Peter Vajda, Sam Tsai, Maryam Daneshi, Matt Yu, Huizhong Chen, Andre Araujo, Bernd Girod Image,
TOWARD AN INTELLIGENT EDITOR FOR JAZZ MUSIC
TOWARD AN INTELLIGENT EDITOR FOR JAZZ MUSIC G.TZANETAKIS, N.HU, AND R.B. DANNENBERG Computer Science Department, Carnegie Mellon University 5000 Forbes Avenue, Pittsburgh, PA 15213, USA E-mail: gtzan@cs.cmu.edu
10GBASE-R Test Patterns
John Ewen jfewen@us.ibm.com Test Pattern Want to evaluate pathological events that occur on average once per day At 1Gb/s once per day is equivalent to a probability of 1.1 1 15 ~ 1/2 5 Equivalent to 7.9σ
Laughbot: Detecting Humor in Spoken Language with Language and Audio Cues
Laughbot: Detecting Humor in Spoken Language with Language and Audio Cues Kate Park, Annie Hu, Natalie Muenster Email: katepark@stanford.edu, anniehu@stanford.edu, ncm000@stanford.edu Abstract We propose
Experiments with Fisher Data
Experiments with Fisher Data Gunnar Evermann, Bin Jia, Kai Yu, David Mrva Ricky Chan, Mark Gales, Phil Woodland May 16th 2004 EARS STT Meeting May 2004 Montreal Overview Introduction Pre-processing 2000h
DAY 1. Intelligent Audio Systems: A review of the foundations and applications of semantic audio analysis and music information retrieval
DAY 1 Intelligent Audio Systems: A review of the foundations and applications of semantic audio analysis and music information retrieval Jay LeBoeuf Imagine Research jay{at}imagine-research.com Rebecca
Music Emotion Recognition. Jaesung Lee. Chung-Ang University
Music Emotion Recognition Jaesung Lee Chung-Ang University Introduction Searching Music in Music Information Retrieval Some information about target music is available Query by Text: Title, Artist, or
Frame Processing Time Deviations in Video Processors
Tensilica White Paper Frame Processing Time Deviations in Video Processors May, 2008 1 Executive Summary Chips are increasingly made with processor designs licensed as semiconductor IP (intellectual property).
Available online at ScienceDirect. Procedia Computer Science 46 (2015 )
Available online at www.sciencedirect.com ScienceDirect Procedia Computer Science 46 (2015 ) 381 387 International Conference on Information and Communication Technologies (ICICT 2014) Music Information
AUTOMATIC ACCOMPANIMENT OF VOCAL MELODIES IN THE CONTEXT OF POPULAR MUSIC
AUTOMATIC ACCOMPANIMENT OF VOCAL MELODIES IN THE CONTEXT OF POPULAR MUSIC A Thesis Presented to The Academic Faculty by Xiang Cao In Partial Fulfillment of the Requirements for the Degree Master of Science
Towards Using Hybrid Word and Fragment Units for Vocabulary Independent LVCSR Systems
Towards Using Hybrid Word and Fragment Units for Vocabulary Independent LVCSR Systems Ariya Rastrow, Abhinav Sethy, Bhuvana Ramabhadran and Fred Jelinek Center for Language and Speech Processing IBM TJ
Pitch-Gesture Modeling Using Subband Autocorrelation Change Detection
Published at Interspeech 13, Lyon France, August 13 Pitch-Gesture Modeling Using Subband Autocorrelation Change Detection Malcolm Slaney 1, Elizabeth Shriberg 1, and Jui-Ting Huang 1 Microsoft Research,
Computer Coordination With Popular Music: A New Research Agenda 1
Computer Coordination With Popular Music: A New Research Agenda 1 Roger B. Dannenberg roger.dannenberg@cs.cmu.edu http://www.cs.cmu.edu/~rbd School of Computer Science Carnegie Mellon University Pittsburgh,
A repetition-based framework for lyric alignment in popular songs
A repetition-based framework for lyric alignment in popular songs ABSTRACT LUONG Minh Thang and KAN Min Yen Department of Computer Science, School of Computing, National University of Singapore We examine
A probabilistic framework for audio-based tonal key and chord recognition
A probabilistic framework for audio-based tonal key and chord recognition Benoit Catteau 1, Jean-Pierre Martens 1, and Marc Leman 2 1 ELIS - Electronics & Information Systems, Ghent University, Gent (Belgium)
A Framework for Segmentation of Interview Videos
A Framework for Segmentation of Interview Videos Omar Javed, Sohaib Khan, Zeeshan Rasheed, Mubarak Shah Computer Vision Lab School of Electrical Engineering and Computer Science University of Central Florida
Outline. Why do we classify? Audio Classification
Outline Introduction Music Information Retrieval Classification Process Steps Pitch Histograms Multiple Pitch Detection Algorithm Musical Genre Classification Implementation Future Work Why do we classify
SIDRA INTERSECTION 8.0 UPDATE HISTORY
Akcelik & Associates Pty Ltd PO Box 1075G, Greythorn, Vic 3104 AUSTRALIA ABN 79 088 889 687 For all technical support, sales support and general enquiries: support.sidrasolutions.com SIDRA INTERSECTION
A Transfer Learning Based Feature Extractor for Polyphonic Sound Event Detection Using Connectionist Temporal Classification
INTERSPEECH 17 August, 17, Stockholm, Sweden A Transfer Learning Based Feature Extractor for Polyphonic Sound Event Detection Using Connectionist Temporal Classification Yun Wang and Florian Metze Language
Design Project: Designing a Viterbi Decoder (PART I)
Digital Integrated Circuits A Design Perspective 2/e Jan M. Rabaey, Anantha Chandrakasan, Borivoje Nikolić Chapters 6 and 11 Design Project: Designing a Viterbi Decoder (PART I) 1. Designing a Viterbi
Sentiment Analysis. Andrea Esuli
Sentiment Analysis Andrea Esuli What is Sentiment Analysis? What is Sentiment Analysis? Sentiment analysis and opinion mining is the field of study that analyzes people s opinions, sentiments, evaluations,
PREDICTING HUMOR RESPONSE IN DIALOGUES FROM TV SITCOMS. Dario Bertero, Pascale Fung
PREDICTING HUMOR RESPONSE IN DIALOGUES FROM TV SITCOMS Dario Bertero, Pascale Fung Human Language Technology Center The Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong dbertero@connect.ust.hk,
Introduction to Sentiment Analysis. Text Analytics - Andrea Esuli
Introduction to Sentiment Analysis Text Analytics - Andrea Esuli What is Sentiment Analysis? What is Sentiment Analysis? Sentiment analysis and opinion mining is the field of study that analyzes people
Music Recommendation from Song Sets
Music Recommendation from Song Sets Beth Logan Cambridge Research Laboratory HP Laboratories Cambridge HPL-2004-148 August 30, 2004* E-mail: Beth.Logan@hp.com music analysis, information retrieval, multimedia
Reducing False Positives in Video Shot Detection
Reducing False Positives in Video Shot Detection Nithya Manickam Computer Science & Engineering Department Indian Institute of Technology, Bombay Powai, India - 400076 mnitya@cse.iitb.ac.in Sharat Chandran
A Novel Bus Encoding Technique for Low Power VLSI
A Novel Bus Encoding Technique for Low Power VLSI Jayapreetha Natesan and Damu Radhakrishnan * Department of Electrical and Computer Engineering State University of New York 75 S. Manheim Blvd., New Paltz,
Quick Reference Manual
Quick Reference Manual V1.0 1 Contents 1.0 PRODUCT INTRODUCTION...3 2.0 SYSTEM REQUIREMENTS...5 3.0 INSTALLING PDF-D FLEXRAY PROTOCOL ANALYSIS SOFTWARE...5 4.0 CONNECTING TO AN OSCILLOSCOPE...6 5.0 CONFIGURE
Retrieval of textual song lyrics from sung inputs
INTERSPEECH 2016 September 8 12, 2016, San Francisco, USA Retrieval of textual song lyrics from sung inputs Anna M. Kruspe Fraunhofer IDMT, Ilmenau, Germany kpe@idmt.fraunhofer.de Abstract Retrieving the
Personalized TV Recommendation with Mixture Probabilistic Matrix Factorization
Personalized TV Recommendation with Mixture Probabilistic Matrix Factorization Huayu Li, Hengshu Zhu #, Yong Ge, Yanjie Fu +,Yuan Ge Computer Science Department, UNC Charlotte # Baidu Research-Big Data
Laughter Valence Prediction in Motivational Interviewing based on Lexical and Acoustic Cues
Laughter Valence Prediction in Motivational Interviewing based on Lexical and Acoustic Cues Rahul Gupta o, Nishant Nath, Taruna Agrawal o, Panayiotis Georgiou, David Atkins +, Shrikanth Narayanan o o Signal
Deep Learning of Audio and Language Features for Humor Prediction
Deep Learning of Audio and Language Features for Humor Prediction Dario Bertero, Pascale Fung Human Language Technology Center Department of Electronic and Computer Engineering The Hong Kong University
MODELS of music begin with a representation of the
602 IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 18, NO. 3, MARCH 2010 Modeling Music as a Dynamic Texture Luke Barrington, Student Member, IEEE, Antoni B. Chan, Member, IEEE, and
MC9211 Computer Organization
MC9211 Computer Organization Unit 2 : Combinational and Sequential Circuits Lesson2 : Sequential Circuits (KSB) (MCA) (2009-12/ODD) (2009-10/1 A&B) Coverage Lesson2 Outlines the formal procedures for the
AMERICAN NATIONAL STANDARD
Digital Video Subcommittee AMERICAN NATIONAL STANDARD ANSI/SCTE 197 2018 Recommendations for Spot Check Loudness Measurements NOTICE The Society of Cable Telecommunications Engineers (SCTE) / International
Seminar CHIST-ERA Istanbul : 4 March 2014 Kick-off meeting : 27 January 2014 (call IUI 2012)
project JOKER JOKe and Empathy of a Robot/ECA: Towards social and affective relations with a robot Seminar CHIST-ERA Istanbul : 4 March 2014 Kick-off meeting : 27 January 2014 (call IUI 2012) http://www.chistera.eu/projects/joker
CS 1674: Intro to Computer Vision. Face Detection. Prof. Adriana Kovashka University of Pittsburgh November 7, 2016
CS 1674: Intro to Computer Vision Face Detection Prof. Adriana Kovashka University of Pittsburgh November 7, 2016 Today Window-based generic object detection basic pipeline boosting classifiers face detection
Time Series Models for Semantic Music Annotation Emanuele Coviello, Antoni B. Chan, and Gert Lanckriet
IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 19, NO. 5, JULY 2011 1343 Time Series Models for Semantic Music Annotation Emanuele Coviello, Antoni B. Chan, and Gert Lanckriet Abstract
MUSIC TONALITY FEATURES FOR SPEECH/MUSIC DISCRIMINATION. Gregory Sell and Pascal Clark
214 IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP) MUSIC TONALITY FEATURES FOR SPEECH/MUSIC DISCRIMINATION Gregory Sell and Pascal Clark Human Language Technology Center
Jazz Melody Generation and Recognition
Jazz Melody Generation and Recognition Joseph Victor December 14, 2012 Introduction In this project, we attempt to use machine learning methods to study jazz solos. The reason we study jazz in particular
DETECTION OF SLOW-MOTION REPLAY SEGMENTS IN SPORTS VIDEO FOR HIGHLIGHTS GENERATION
DETECTION OF SLOW-MOTION REPLAY SEGMENTS IN SPORTS VIDEO FOR HIGHLIGHTS GENERATION H. Pan P. van Beek M. I. Sezan Electrical & Computer Engineering University of Illinois Urbana, IL 6182 Sharp Laboratories
Detection and demodulation of non-cooperative burst signal Feng Yue 1, Wu Guangzhi 1, Tao Min 1
International Conference on Applied Science and Engineering Innovation (ASEI 2015) Detection and demodulation of non-cooperative burst signal Feng Yue 1, Wu Guangzhi 1, Tao Min 1 1 China Satellite Maritime
Week 14 Query-by-Humming and Music Fingerprinting. Roger B. Dannenberg Professor of Computer Science, Art and Music Carnegie Mellon University
Week 14 Query-by-Humming and Music Fingerprinting Roger B. Dannenberg Professor of Computer Science, Art and Music Overview n Melody-Based Retrieval n Audio-Score Alignment n Music Fingerprinting 2 Metadata-based
IMPROVED MELODIC SEQUENCE MATCHING FOR QUERY BASED SEARCHING IN INDIAN CLASSICAL MUSIC
IMPROVED MELODIC SEQUENCE MATCHING FOR QUERY BASED SEARCHING IN INDIAN CLASSICAL MUSIC Ashwin Lele #, Saurabh Pinjani #, Kaustuv Kanti Ganguli, and Preeti Rao Department of Electrical Engineering, Indian
Searching for Similar Phrases in Music Audio
Searching for Similar Phrases in Music udio an Ellis Laboratory for Recognition and Organization of Speech and udio ept. Electrical Engineering, olumbia University, NY US http://labrosa.ee.columbia.edu/
UNIT IV. Sequential circuit
UNIT IV Sequential circuit Introduction In the previous session, we said that the output of a combinational circuit depends solely upon the input. The implication is that combinational circuits have no
Audio Feature Extraction for Corpus Analysis
Audio Feature Extraction for Corpus Analysis Anja Volk Sound and Music Technology 5 Dec 2017 1 Corpus analysis What is corpus analysis study a large corpus of music for gaining insights on general trends
(12) United States Patent (10) Patent No.: US 6,628,712 B1
USOO6628712B1 (12) United States Patent (10) Patent No.: Le Maguet (45) Date of Patent: Sep. 30, 2003 (54) SEAMLESS SWITCHING OF MPEG VIDEO WO WP 97 08898 * 3/1997... HO4N/7/26 STREAMS WO WO990587O 2/1999...
19 th INTERNATIONAL CONGRESS ON ACOUSTICS MADRID, 2-7 SEPTEMBER 2007
19 th INTERNATIONAL CONGRESS ON ACOUSTICS MADRID, 2-7 SEPTEMBER 2007 AN HMM BASED INVESTIGATION OF DIFFERENCES BETWEEN MUSICAL INSTRUMENTS OF THE SAME TYPE PACS: 43.75.-z Eichner, Matthias; Wolff, Matthias;
Sequencing and Control
Sequencing and Control Lan-Da Van ( 范倫達 ), Ph. D. Department of Computer Science National Chiao Tung University Taiwan, R.O.C. Spring, 2016 ldvan@cs.nctu.edu.tw http://www.cs.nctu.edu.tw/~ldvan/ Source:
Efficient Computer-Aided Pitch Track and Note Estimation for Scientific Applications. Matthias Mauch Chris Cannam György Fazekas
Efficient Computer-Aided Pitch Track and Note Estimation for Scientific Applications Matthias Mauch Chris Cannam György Fazekas! 1 Matthias Mauch, Chris Cannam, George Fazekas Problem Intonation in Unaccompanied
A fragment-decoding plus missing-data imputation ASR system evaluated on the 2nd CHiME Challenge
A fragment-decoding plus missing-data imputation ASR system evaluated on the 2nd CHiME Challenge Ning Ma MRC Institute of Hearing Research, Nottingham, NG7 2RD, UK n.ma@ihr.mrc.ac.uk Jon Barker Department
A Low-Power 0.7-V H p Video Decoder
A Low-Power 0.7-V H.264 720p Video Decoder D. Finchelstein, V. Sze, M.E. Sinangil, Y. Koken, A.P. Chandrakasan A-SSCC 2008 Outline Motivation for low-power video decoders Low-power techniques pipelining
A STATISTICAL VIEW ON THE EXPRESSIVE TIMING OF PIANO ROLLED CHORDS
A STATISTICAL VIEW ON THE EXPRESSIVE TIMING OF PIANO ROLLED CHORDS Mutian Fu 1 Guangyu Xia 2 Roger Dannenberg 2 Larry Wasserman 2 1 School of Music, Carnegie Mellon University, USA 2 School of Computer
MUSI-6201 Computational Music Analysis
MUSI-6201 Computational Music Analysis Part 9.1: Genre Classification alexander lerch November 4, 2015 temporal analysis overview text book Chapter 8: Musical Genre, Similarity, and Mood (pp. 151 155)
Bi-Modal Music Emotion Recognition: Novel Lyrical Features and Dataset
Bi-Modal Music Emotion Recognition: Novel Lyrical Features and Dataset Ricardo Malheiro, Renato Panda, Paulo Gomes, Rui Paiva CISUC Centre for Informatics and Systems of the University of Coimbra {rsmal,
GYROPHONE RECOGNIZING SPEECH FROM GYROSCOPE SIGNALS. Yan Michalevsky (1), Gabi Nakibly (2) and Dan Boneh (1)
GYROPHONE RECOGNIZING SPEECH FROM GYROSCOPE SIGNALS Yan Michalevsky (1), Gabi Nakibly (2) and Dan Boneh (1) (1) Stanford University (2) National Research and Simulation Center, Rafael Ltd. 0 MICROPHONE
Classification of Timbre Similarity
Classification of Timbre Similarity Corey Kereliuk McGill University March 15, 2007 1 / 16 1 Definition of Timbre What Timbre is Not What Timbre is A 2-dimensional Timbre Space 2 3 Considerations Common
Dual frame motion compensation for a rate switching network
Dual frame motion compensation for a rate switching network Vijay Chellappa, Pamela C. Cosman and Geoffrey M. Voelker Dept. of Electrical and Computer Engineering, Dept. of Computer Science and Engineering
Acoustic Prosodic Features In Sarcastic Utterances
Acoustic Prosodic Features In Sarcastic Utterances Introduction: The main goal of this study is to determine if sarcasm can be detected through the analysis of prosodic cues or acoustic features automatically.
Video Transmission. Thomas Wiegand: Digital Image Communication Video Transmission 1. Transmission of Hybrid Coded Video. Channel Encoder.
Video Transmission Transmission of Hybrid Coded Video Error Control Channel Motion-compensated Video Coding Error Mitigation Scalable Approaches Intra Coding Distortion-Distortion Functions Feedback-based
Section 001. Read this before starting!
Points missed: Student's Name: Total score: / points East Tennessee State University epartment of Computer and Information Sciences CSCI 25 (Tarnoff) Computer Organization TEST 2 for Spring Semester, 23
CS 591 S1 Computational Audio
4/29/7 CS 59 S Computational Audio Wayne Snyder Computer Science Department Boston University Today: Comparing Musical Signals: Cross- and Autocorrelations of Spectral Data for Structure Analysis Segmentation
Sarcasm Detection in Text: Design Document
CSC 59866 Senior Design Project Specification Professor Jie Wei Wednesday, November 23, 2016 Sarcasm Detection in Text: Design Document Jesse Feinman, James Kasakyan, Jeff Stolzenberg 1 Table of contents
A Comparative Study of Spectral Transformation Techniques for Singing Voice Synthesis
INTERSPEECH 2014 A Comparative Study of Spectral Transformation Techniques for Singing Voice Synthesis S. W. Lee 1, Zhizheng Wu 2, Minghui Dong 1, Xiaohai Tian 2, and Haizhou Li 1,2 1 Human Language Technology
Analogue Versus Digital [5 M]
![Analogue Versus Digital [5 M]](/thumbs/93/111640168.jpg "Analogue Versus Digital [5 M]")
Q.1 a. Analogue Versus Digital [5 M] There are two basic ways of representing the numerical values of the various physical quantities with which we constantly deal in our day-to-day lives. One of the ways,
Perceptual dimensions of short audio clips and corresponding timbre features
Perceptual dimensions of short audio clips and corresponding timbre features Jason Musil, Budr El-Nusairi, Daniel Müllensiefen Department of Psychology, Goldsmiths, University of London Question How do
Signal Persistence Checking of Asynchronous System Implementation using SPIN
, March 18-20, 2015, Hong Kong Signal Persistence Checking of Asynchronous System Implementation using SPIN Weerasak Lawsunnee, Arthit Thongtak, Wiwat Vatanawood Abstract Asynchronous system is widely
IEEE TRANSACTIONS ON SPEECH AND AUDIO PROCESSING 1. Note Segmentation and Quantization for Music Information Retrieval
IEEE TRANSACTIONS ON SPEECH AND AUDIO PROCESSING 1 Note Segmentation and Quantization for Music Information Retrieval Norman H. Adams, Student Member, IEEE, Mark A. Bartsch, Member, IEEE, and Gregory H.