Starter tips for automating and monitoring flows with Tableau Prep Conductor

Historically, data prep was an IT function. Today, many organizations are adopting self-service data prep solutions to explore and prototype new data sources and use cases that complement their modern analytics. That’s why we released Tableau Prep Builder last spring. But customers wanted to schedule and monitor their prep flows—so with our 2019.1 release, we introduced Tableau Prep Conductor to extend the data prep capabilities of the Tableau platform.

Now, users can automate data prep flows to run without manual updates as the data changes, and manage prepped data through their Tableau server environment. This means more visibility into data prep at scale, and clean data is more discoverable to the organization.

If you’re just getting started with Tableau Prep Conductor, here are some helpful tips for using some of the new features to manage self-service data prep at scale. Please note: Because Tableau Online is a hosted environment, some tips will apply only to Tableau Server.

Set flows to run automatically during non-work hours.

Schedule prep flow updates during non-work hours, like at night, when there are fewer jobs competing for resources. Sparing an employee from manually updating the data will not only save time, but you can also take advantage of a stable server environment to run the flow updates, rather than someone using their own desktop resources.

A flow task for a recurring schedule to automatically update a prep flow.

Automate additional functionality with the REST API.

You can use the REST API to build workflows with third-party systems and connect your data pipeline in powerful, flexible ways—including updating flow connections, managing permissions, and much more. If your flow refresh is dependent on other back-end system jobs finishing first, you may not be able to schedule an exact time for a refresh. Instead, use the REST API to automatically trigger a flow run when the other processes are completed.

Learn more about your flow performance with Administrative Views.

With Tableau Prep Conductor, you can monitor flows with the same tools available on Tableau Server today—which includes out-of-the-box Administrative Views that can help answer important questions about your organization’s data prep practices. You’ll find views tracking everything from scheduled vs. ad hoc flows, the durations of tasks, actions by users, stats for space usage, and more.

An Administrative View showing actions taken by a user on a particular server.

Keep an eye on server statuses.

The Tableau Server Status page and Tableau Services Manager status page include Tableau Server processes, along with links to troubleshooting documentation if a process is not running as expected. If you hover your mouse pointer over the status indicator for a process, a tooltip shows the node name and the port the process is running on, so you know exactly what to investigate or address.

Use alerts to take action fast.

You can receive a notification within the Server interface as well as via email, should a flow process fail. Links within the alert will point you to quickly take action to examine errors, fix them with suggested remedies, and get back to your prep or analysis.

Alerts for a failed flow run, shown in email and within Tableau Online.

Use appropriate tagging to ensure flows are discoverable.

Just like you do today with workbooks, you can apply keywords to flows help users find, filter, and categorize content. This is helpful when sharing work across the organization, so flows can be found and reused, or modified for similar use cases, rather than recreated from scratch. And of course, you can add tags to multiple flows at once.

Check for correct permissions and authentication.

For prep flows managed by Tableau Prep Conductor, you can set permissions for a flow like other analytics content in your server environment—who can see it, edit it, run the flow, and more. If a flow connects to databases, you can also specify the authentication type and set credentials to access the data when you publish the flow.

Permissions options for a prep flow

Self-service data prep is still a brand-new skill set for many and needs to be developed and rolled out thoughtfully for success at scale. Investing in a governance framework and education will help employees understand and perform prep functions effectively—including establishing repeatable processes, automating them for efficiency, and ultimately building trust and confidence in the data produced.

These needs should be assessed in your organization based on the roles and volume of people who will use Tableau Prep Builder and Tableau Prep Conductor for self-service data prep. To learn more helpful tips and dive deeper into other organizational considerations, check out our whitepaper, Managing the data prep chaos in your organization.