How PepsiCo tamed big data and cut analysis time by 70%

PepsiCo relies on huge volumes of data to accurately supply its retailers in more than 200 countries to serve a billion customers every day.

Supplying too much product results in wasted resources. Supplying too little means risking profit loss—and the company must reconcile with unhappy, empty-handed retailers. An empty shelf also risks consumers choosing a competitor's product, which has harmful, long-term effects on the brand.

To strike the right balance between appropriate product-stocking levels and razor-thin margins, PepsiCo continually refines sales forecasts.

As an analyst in PepiCo’s Collaborative Planning, Forecasting, and Replenishment (CPFR) team, Mike Riegling provides data insights that enable effective retail-sales management. Mike's team also collaborates with large retailers to supply the right quantities of product for their warehouses and stores.

The journey to analytics success was not easy. There were many hurdles along the way. But by using Trifacta to wrangle disparate data, Mike and his team reduced end-to-end run time of analysis by as much as 70 percent. And with Tableau, the team cut report production time as much as 90 percent.

“It used to take an analyst 90 minutes to create a report on any given day. Now it takes less than 20 minutes,” says Mike.

Searching for a single source of truth

PepsiCo’s customers provide reports that include warehouse inventory, store inventory, and point-of-sale inventory. PepsiCo then reconciles this data with its own shipment history, production numbers, and forecast data.

Each customer had its own data standards, which didn't correspond with each other, let alone PepsiCo's system. For example, PepsiCo relies on UPC codes to identify each product while customers create their own internal numbers.

Wrangling this data was a challenge. Reports could take months to generate. Analysts knew their ability to quickly standardize data across all retailers—and speak the same language as their customers—was critical to preparing data faster for their forecasting and planning efforts.

Churning out sales forecasts fast enough for management to steer the course on sales was another challenge. Each new report required the CPFR team’s analysts to build a tool in Microsoft Access that combined retailers’ sales data and PepsiCo supply data—a process that could take up to six months.

The team primarily relied on Excel for analysis, creating large quantities of messy data. And the team had no efficient way to spot errors, leading to potentially costly outcomes. A missing product from a report, for example, could result in inaccurate forecasts and lost revenue.

The CPFR team needed a way to wrangle large quantities of disparate data. At the same time, the team needed a visual analytics tool that could help the team make the most of PepsiCo data.

Finding the right solution for better data

To bring coherence to the data, PepsiCo turned to Tableau partner Trifacta. For greater efficiency and scale, PepsiCo implemented Hortonworks Hadoop as a landing and staging environment for this variety of data.

Now reports run directly on Hadoop without involving multiple steps with Access and PepsiCo servers. This process allows analysts to directly manipulate data using Trifacta.

“This really bridges the gap between business and IT. This technology allows IT to open up access to raw data to business users," says Mike. “Part of the ethos of the PepsiCo organization is selecting technology that will enable the business to self-serve and get IT out of the job of responding to every single question or demand from the business.”

Tableau provides PepsiCo with the finishing step: powerful analytics and interactive visualizations, helping the business uncover insights in volumes of data. PepsiCo analysts can easily share relevant Tableau workbooks with management and the business using Tableau Server.

“Tableau not only serves as the corporate standard for business intelligence—the reporting on top of the enterprise data warehouse—but also as a self-service tool for analysts and departmental uses cases,” says Mike.

Enabling powerful results with self-service analytics

The combined use of Tableau, Hortonworks, and Trifacta helps the CPFR team drive the business forward, increasing visibility into customer orders.

Trifacta integrates natively with Tableau Data Extracts (TDE), allowing PepsiCo analysts to seamlessly transition between the two technologies. The IT department publishes TDEs to Tableau Server, encouraging collaboration and enabling self-service analytics at scale. PepsiCo brings Tableau Data Extracts into Tableau Desktop all while maintaining performance on big data.

This has dramatically reduced time spent on data preparation and increased the overall quality of the data. Normalized data in Trifacta helps catch errors and flaws in the data. And with Tableau, PepsiCo can easily spot outliers and take action. If issues arise, they can correct mistakes before they cause bigger problems downstream.

By using Tableau to analyze Trifacta's normalized data, PepsiCo has reduced the end-to-end run time of analysis by as much as 70 percent. The ability to reproduce patterns makes building CPFR tools a more efficient, automated process.

Building these reports in Tableau reduces production time by as much as 90 percent, and dashboards that used to take up to six months to build are now ready within a day. The company can now analyze inventory, logistical, and finance data from across the organization to create supply-chain and forecasting reports in record time.

“Now we’re not taking our time putting the data together; we’re spending our time analyzing the data, telling the story, and finding problems,” Mike says.

PepsiCo can now predict trends, adjust sales, and ultimately sell more of the right product to improve their bottom line.

“We’re able to turn customer data around and present it to the rest of our company, in a way that everyone can understand, faster than our competitors,” says Mike.

To learn more about how PepsiCo is enabling enterprise self-service analytics at scale, check out Mike’s recent webinar. And to learn more about finding stories in your big data, visit our big-data analytics page.

Headed to Strata - Hadoop World? Stop by our booth (#762) for a live Tableau demo with big data.