Twitter analytics: Thinking beyond sentiments
Internet is flooded with numerous articles and tutorials on twitter data analytics. Many of them talk about how to use Twitter developer APIs to fetch tweets. And do sentiment analytics!
But is that all we can do with tweets? If No, then why everyone is behind sentiments? Because it is easy to begin. And I get that. Libraries like Textblob, Spacy come with built-in sentiment classification.
Before going to sentiments there is enough exploratory analysis that can be performed to get relevant insights. In this post, let's attempt and shift our attention to some other stuff. Some simple operations that beginners/learners can perform when they are thrown with Twitter data to analyze.
So what can we do? Clean your shit first!
Start with Cleaning your data. Basic parsing can be done using this package. You can then separate smilies, emojis, URLs, hashtags, numbers. If you have metadata like retweets, user ID, etc, you can separate that as well.
See the below function which accepts tweets data-frame and returns the same with additional columns of separated features.
Most common topics and feelings
Simple descriptive analysis will give you a good understanding of the data. Below snippet gives most used hashtags, emojis, smileys
We can then focus on important tweets with hashtags/emojis as a filter
Word clouds with Twist!
Finally, we will use descriptive analytics methodology to generate word clouds. But wait, again everyone does that. All the time. No?
We can give it a twist.
Use NER(named entity recognition) to create meaningful word clouds rich in context and intuitive to visualize.
Contextual masks can be used to visualize word clouds. Below image shows various outputs that can be generated.
Bonus Step
Cleansed data ignites the imaginative minds of data scientists.
One can perform Troll Detection. We can use smilies and hashtags as features to train a troll classifier. Imagine a tweet with some serious hashtag followed by 😬 😁 😂. Quite a potential to mark as a troll. Isn’t it?
