TextWise

Recently we’ve been looking at how well our Semantic Signatures technology works with messages posted to Twitter. These kinds of messages pose significant challenges for the semantic web in general, because their extremely short length (140 characters or less) means that there will be very little context available for understanding the content of the message. In addition, many of these messages feature “creative” spellings and grammar, and are of a personal nature (e.g. “Having sushi for lunch today”) that would not be of general interest. Extracting any meaningful information from these snippets of random conversation will be quite a difficult task indeed.

To see what exactly we’re up against, we undertook a small study to characterize the different types of messages that can be found on Twitter. We downloaded a sample of tweets over a two-week period using the Twitter streaming API. This resulted in a corpus of 8.9 million messages (“tweets”) posted by 2.6 million unique users. About 2.7 million of these tweets, or 31%, were replies to a tweet posted by another user, while half a million (6%) were retweets. Almost 2 million (22%) of the messages contained a URL.

Next we used a modified version of the language-guessing program TextCat to find the distribution of languages in our corpus.

As you might expect, English turns out to be the most common language used on Twitter (61%). But surprisingly, the next most common language is Portuguese (11%), beating out both Japanese (6%) and Spanish (4%). This is the opposite ordering of these three languages compared to what we find on the internet in general (http://www.internetworldstats.com/stats7.htm). It seems that Twitter must be a lot more popular in Brazil than in the rest of Latin America. Other languages like French, German, and even Malay are also fairly common in our corpus. It’s hard to get accurate counts for these, though, because TextCat doesn’t deal well with shorter texts (note the estimated 10% of messages that are unknown or misclassified), and it doesn’t seem to be able to identify these languages in our sample as reliably as the first four languages.

We were then interested in seeing what kinds of messages get posted to Twitter. Is it really all just people talking about where they are right now and what they’re having for lunch, or is there actually some informative content out there for us to find? To answer this question, we took a random sample of 1,000 English-language tweets from the corpus we collected, and then classified each message as one of the following types:

• User’s current status — where the user is right now, what they’re doing, etc.
• Private conversations — some twitterers seem to use the service as if it were a giant internet chatroom
• Links to web content — a URL with an article title and/or some commentary on its content. Further broken down into: links to blog and news articles; links to images and videos; and other links.
• Politics, sports, current events — discussion of these topics
• Product recommendations/complaints — recommendations or complaints about specific TV shows, movies, techie gadgets, etc.
• Advertising — posted from a company’s twitter account
• Spam — a strange phenomenon, given that an account has to be followed for anyone to see its tweets, but it does exist
• Other messages — messages that don’t quite fit under any of the above categories. Fan messages to celebrities, shoutouts to other users, web-based polls and quizzes, and so on.

Here is a graph showing the frequency of these different message types in the 1,000 tweet sample.

As you can see, over half of the tweets in the sample are either user statuses or private conversations. Only about 10-20% of the messages could be considered more broadly relevant to a larger audience (depending on which message types are considered to be informative). So while there is some interesting content to be discovered on Twitter, it will definitely take a bit of work to find it.

We also collected some statistics on the twitterers in our sample, such as their average rate of posting, the most common clients they use, and so on. But this post is already pretty long, so I’ll save that for a later date. Stay tuned…

Tags: corpus analysis, pragmatics, speech acts, twitter

This entry was posted on Friday, January 8th, 2010 at 3:28 pm and is filed under Science. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.