What Twitter's Data Changes Mean for Social Media Marketers

For social media marketers, the world is full of choices—often too many choices. Which hashtag should I use? When should I post that Facebook update? What’s the best tool for collecting data?

Our social media team is constantly experimenting to see what’s possible with data. Not only do we analyze our own social media channels, we also use social data to explore broader conversations.

Over the past year, we’ve pushed ourselves to ask deeper questions, especially about conversations on Twitter. Using data collected from Tableau partner DataSift, we’ve analyzed World Cup hashtags, SXSW session popularity, and journalists' use of Twitter during the UK general election.

Do media accounts engage with people on Twitter? Which of the smaller political parties get mentioned the most? These are the types of questions we explored during the recent UK election.

You've probably heard about the recent changes in Twitter’s data ecosystem. Twitter is in the process of transitioning data access directly to its own platform. By mid-August, raw Twitter data will no longer be available through a reseller, which means the type of Twitter analysis we’ve been exploring won't be possible with DataSift alone. (DataSift is, however, working on ways to post-process data that you obtain from Twitter.)

If you’re working with other social data, DataSift remains an excellent platform with both historical and live data for other social networks, news sites, and blogs. The company also recently partnered with Facebook to give marketers a deeper understanding of the topics that people are engaging in on Facebook. DataSift is currently the only company offering access to aggregated and anonymized Facebook topic data.

We’ve also been working with Twitter on some new resources for getting data into Tableau (more information below), and we’re excited to see Twitter go deep into the world of data. But we also know these changes will create disruptions (not to mention even more choices) in the everyday lives of social media marketers, ourselves included.

So if you currently get Twitter data through a reseller, we want to help you understand all your options—so you can keep focusing on analysis and innovation.

Here are three main options:

Transition to Twitter’s Gnip Platform for Firehose Access

In August, Gnip will become the sole provider of raw Twitter data for both live and historic Tweets. This includes programmatic API access for real-time filtering, sampling, and historical analysis of Tweets. Similar to DataSift, Gnip also connects to and normalizes data from other social networks, not just Twitter.

We know that working with APIs is not for everyone. So we’ve worked closely with @TwitterDev to support an open-source connector they built for non-technical users to create filter rules and stream data directly into Google BigQuery. This can serve as the source for your Tableau dashboards.

If you are using existing data sources, you will unfortunately need to rebuild the fields in your dashboards. However, documentation on the filter options for Gnip’s API (called PowerTrack) as well as the data format are available on Gnip’s support site.

Export Data from Twitter or a Third Party

Twitter already provides a basic activity dashboard for its users. By exporting that data and bringing it into Tableau, you can work with it more flexibly to answer your own questions.

Keep in mind that this data has no sentiment analysis or other advanced listening. (Also, CSV downloads are limited to 50,000 records.) However, that doesn’t mean you can’t get powerful insights from this data, even if it’s just adding context to other data.

Many third-party marketing tools also let you export data as a CSV. This can be a great option if the CSVs are well-formatted and reasonable to maintain. After all, flat files will always require some manual work, but we use them a lot on our own social media team.

Try the Twitter API

With the Twitter API, you can access a small sample of live Tweets for free. This can be a great option if want raw data and don’t need the full level of access that Gnip provides (see above).

You will need some coding chops to access the Twitter API and transform the data into a database. However, learning the API will certainly help if you want to get a better understanding of Gnip’s API, PowerTrack.

The bottom line is that social media is evolving fast and the tools to access that data are changing as well. Remember that you can always blend data sources in Tableau. So if you use flat files for Pinterest, the Twitter API for Tweets, and DataSift for aggregated and anonymized Facebook topic data, you can still see all your insights on a single dashboard.

Twitter’s data changes will affect the way we access data. They may even change the questions we ask once we get that data. But we will continue to ask those questions, adapting our analysis for whatever data source we’re working with.

We look forward to sharing our projects with you, and hearing how they apply to your own data questions.