Laughbot: Detecting Humor in Spoken Language with Language and Audio Cues
We propose detecting and responding to humor in spoken dialogue by extracting language and audio cues and subsequently feeding these features into a combined recurrent neural network (RNN) and logistic regression model. In this paper, we parse Switchboard phone conversations to build a corpus of punchlines and unfunny lines where punchlines precede laughter tokens in Switchboard transcripts. We create a combined RNN and logistic regression model that uses both acoustic and language cues to predict whether a conversational agent should respond to an utterance with laughter. Our model achieves an F1-score of 63.2 and accuracy of 73.9. This model outperforms our logistic language model (F1-score 56.6) and RNN acoustic model (59.4) as well as the final RNN model of D. Bertero, 2016 (52.9). Using our final model, we create a “laughbot” that audibly responds to a user with laughter when their utterance is classified as a punchline. A conversational agent outfitted with a humor-recognition system such as the one we present in this paper would be valuable as these agents gain utility in everyday life.
KeywordsChatbots Spoken natural language processing Deep learning Machine learning
We would like to thank our professor Andrew Maas, Dan Jurafsky and the Stanford University CS244S Spoken Natural Language Processing teaching team. Special thanks to Raghav and Jiwei for their direction on our combined RNN and regression model.
- 1.Nijholt, A.: Humor and embodied conversational agents (2003)Google Scholar
- 2.Piot, M.G.B., Pietquin, O.: Predicting when to laugh with structured classification. In: Interspeech (2014)Google Scholar
- 3.Piot, O.P.B., Geist, M.: Imitation learning applied to embodied conversational agents. In: MLIS (2015)Google Scholar
- 4.Bertero, D.: Deep learning of audio and language features for humor prediction (2016)Google Scholar
- 6.Eyben, F., Weninger, F., Gross, F., Schuller, B.: Recent developments in openSMILE, the munich open-source multimedia feature extractor, pp. 835–838. ACM Multimedia (MM) (2013)Google Scholar