Analysis of Twitter data for postmarketing surveillance in pharmacovigilance

Postmarketing surveillance (PMS) has the vital aim to monitor effects of drugs af- ter release for use by the general pop- ulation, but suffers from under-reporting and limited coverage. Automatic meth- ods for detecting drug effect reports, es- pecially for social media, could vastly in- crease the scope of PMS. Very few auto- matic PMS methods are currently avail- able, in particular for the messy text types encountered on Twitter. In this paper we describe first results for developing PMS methods specifically for tweets. We de- scribe the corpus of 125,669 tweets we have created and annotated to train and test the tools. We find that generic tools per- form well for tweet-level language iden- tification and tweet-level sentiment anal- ysis (both 0.94 F1-Score). For detection of effect mentions we are able to achieve 0.87 F1-Score, while effect-level adverse- vs.-beneficial analysis proves harder with an F1-Score of 0.64. Among other things, our results indicate that MetaMap seman- tic types provide a very promising ba- sis for identifying drug effect mentions in tweets.