How to Create a Web Data Connector: A Beginner's Approach

Géraldine Zanolli is a Product Consultant at Tableau. She has developped an Instagram Web Data Connector and is sharing her learnings here.

When I joined Tableau, being a social media addict, in particular an Instagram fan, I wanted to analyse my social media data using Tableau. Come on, it is crucial to know the best time of day to post THIS awesome selfie.

So, I tried to get my Instagram data in Tableau. I googled “Tableau connection to Instagram”, but found no “out of the box” solution. I could not even export the data to Excel. I almost gave up but I then discovered the Web Data Connector option. A Web Data Connector allows you to connect to a world of data that is available over the web.

It was actually the perfect situation to use a Web Data Connector. Tableau does not have a native Instagram connector, Instagram does not have an ODBC (Open Data Connectivity is a standard application interface for accessing database management systems) driver available, so I could not download the data from Instagram into a supported format. But Instagram does have an API that I could utilise to have access to the data using JavaScript.

You should use a Web Data Connector when:

• Your data is outside of existing connectors
• Your data source does not have an existing ODBC driver
• Your data cannot be downloaded into a supported format
• You want to refresh your data frequently and automatically
• Your data is accessible with JavaScript

The Tableau community already created many Web Data Connectors for different platforms, and shared them publicly. You can find them here. If you can find one already created, all you need to copy and paste the link into Tableau Desktop to connect to your data. At that time though, I could not find one for Instagram.

Instagram has an API that sends back data in JSON. The first step was to understand the API. I like to compare APIs to barmen.

The barman controls the drinks as the API controls the data. The barman is the only access that you have to the drinks, and the API is the only point of access to your data. The API sends back the data you asked for, in the same way the barman serves you the drinks you want according to their recipes.

I used tools like Postman or JSON formatter (on Google Chrome) to test the API before starting to code. The API has a lot of documentation on which request you can send and the response schema.

After reading some online tutorials and blogs like Tableau Junkie, I understood that a Web Data Connector uses JavaScript and HTML. I rolled up my sleeves, gathered my JavaScript knowledge and downloaded the SDK (a Software Development Kit is a set of software development tools that allows the creation of applications for a certain software package) from the website to start building my own Web Data Connector.

The principle of a Web Data Connector is simple. The HTML part is the user interface, what the user sees when he/she uses it. The JavaScript code connects to web data, reads the JSON response, and then passes the information on to Tableau.

First thing to do to create your own Web Data Connector; you need to get GIT and the WDC SDK. You also need a web server ready to host your WDC, like Tomcat or WAMP.

Once you have fulfilled all the requirements, you can find in the SDK some code samples. My advice is not to start your WDC from scratch, but from a pre-built one. For instance, you can use the earthquake WDC as a template for your custom code.

In the SDK, you also have a simulator that allows you to test your WDC before trying it in Tableau. It is very useful. In order to debug, you can add a debug line console.log().

Once everything works fine in the simulator, you can use it in Tableau.

A question I have often been asked is: how long do I need to develop a Web Data Connector? It really depends on your JavaScript skills and the API you are using. It took me 2 weeks to release the first version of the Instagram Web Data Connector. After this, I had to submit an application to Instagram to have full access to the Instagram content.

My advice: start with an easy API that does not require authentication or an API key. My favourite one is Swapi, the Star Wars API.

As you now understand, I am a social media addict and I use the Facebook and Twitter Web Data Connectors often. As of now, there is no Snapchat WDC yet, but it is one that I would like to develop!

Apart from creating WDCs, I like to play with the data in my free time. I recently created a visualisation using the Instagram WDC for a colleague who is a huge fan of the Kardashians, and this visualisation that looks at Trump’s Instagram:

We not only have social media WDCs, we also have a movies' one. The next visualisation is using a TMDb – themoviedb.org WDC created by my colleague Michele Tessari.

Web Data Connectors are a great option. You can get the value of your data into Tableau. You do not need to be a software engineer to build your own, having “Google developer” skills is enough, once you understand the basics and master your API.