Big data has transformed the data storage and data analytics paradigm. Instead of storing only the crucial observations for analysis, more companies are developing ways to store data from multiple business processes in their enterprises. Data storage tools, data warehouses, and data lakes make it possible for companies to store and organize vast amounts of data for future analysis. But how are successful companies using their big data? What can organizations learn from these big data applications? In this article, learn about big data in action and how you can start using your data.
Why do we need big data?
Today’s organizations need big data because it allows them to find insights and trends at scale that would be otherwise difficult or impossible to find. Big data also allows companies to innovate with new analyses or models, including predicting a new behavior or trend. As an example, imagine you want to know more about customers who use a streaming video service. You count how many times people click and watch a video online. You count that information for a month and report the total at month’s end. But then a colleague wants to know what age groups watched the video the most, how long they watched the video, and if they clicked on anything after watching the video. Before big data, you would have had to set up the system for another month to collect these new observations or develop a user experience study. One of the benefits of big data is that you likely already have collected the information from the start to answer any follow-up questions quickly. To be proactive with analysis, companies are collecting viewer data, demographic information, search history, social media conversations, and more.
When do you need big data?
Using big data for day-to-day decisions is no easy task. It requires changing your entire culture of decision-making and investing in new technology so analysts can access that data. Companies need to ensure that you are storing and using only relevant data, because using resources to process unnecessary data can be costly. You need big data when you want to analyze large end-to-end processes such as the customer journey or supply chains. Small- and medium-sized data analytics can complete some of these tasks but often don’t give you a full view of the processes.
The value of small data vs. big data
All relevant data is valuable: small and big data sets offer different benefits and you should know the differences between them and when each is appropriate. Small data might mean looking only at the frequency of baby names by year, instead of by day. This would use a yearly aggregation to vastly reduce the size of the data, while still satisfying yearly analysis. If you wanted to see baby name frequency by day and by hour and by parent name and by parent date of birth, and so on — then the data would become much larger to accommodate the more detailed questions and analysis. In practice, big data often looks like storing the information from the loyalty card customers, such as demographic information, and keeping track of their transactions for the year. You could leverage that information to find trends and target messaging to those customers and make recommendations for future purchases. A small dataset here might only look at the number of transactions and profit per day per store for a high level view. If you only need that high level view, your smaller, simpler data might be the perfect solution. If you need to dive into deeper analysis to ask follow up questions, you’ll need richer, bigger data with more dimensions.
Create beautiful visualizations with your data.
Real-world examples of big data applications
Several industries collect and use their large amounts of data to reach their goals, including to:
- Scale-up and down staffing by analyzing seasonal trends
- Increase efficiency by using monitoring data to find bottlenecks in processes
- Make decisions using near real-time data
- Find opportunities for new directions for their businesses by seeing a holistic view of their data
- Identify outliers that might obscure the real story
We’ll review three examples in particular who use big data technologies in new ways.
Example of big data use in healthcare
Organization: Providence St. Joseph Industry: Healthcare Use case: Companies now have access to new sources of unstructured or raw data. In the healthcare industry, patient care organizations can integrate sensor data from patient-monitoring systems to improve alert predictability. They can also use weather and seasonal data to predict staffing and bed needs. In this example, big data technologies allowed Providence St. Joseph to see a comprehensive and holistic view of its disparate data sources to improve patient care and reduce costs. Providence identified the universal problems that could be solved with data, creating unified views to highlight best practices and help reduce waste. Wasteful practices included using unnecessarily costly supplies and medications which can drive up the cost per patient case and make healthcare less affordable for patients in need. Analysts at Providence built dashboards accessible to the entire hospital system, displaying detailed quality data and cost data. The dashboards allow practitioners and clinicians to see analytics that pertain to every hospital, clinician and individual nursing unit. This data transparency has been associated with substantial improvements in quality measures and large reductions in cost of care.
Example of big data use in banking and finance
Organization: MoneySQ Industry: Financial Technology Use case: Big data technologies empower companies to use near real-time or streaming data for analysis. Financial institutions have access to transaction data, using predictive analytics to predict purchase behavior, identify outliers, and alert users to fraud. Also, near real-time data can assist in analyzing market changes for loan risk assessments. Read this case study from MoneySQ to learn how they improved collaboration, communication, and decision-making with the ability to use near real-time data. Before leveraging big data with Tableau, MoneySQ employees tracked their business targets manually, entering duplicated versions of data into multiple platforms like Excel and other online tools. The process of collecting relevant data and submitting reports to management took at least two days, which meant that the company could only make decisions and roll out strategy changes on a weekly basis. Such delays are a huge liability in a fast-paced and ever-changing financial landscape. “But now that we’ve got Tableau, people are making decisions on a day-to-day basis,” shared MoneySQ Chief Data Officer Jacob Wai. “We have become much more versatile in our product offerings to our customers.”
Example of big data use in manufacturing
Organization: Coopers Brewery Industry: Manufacturing Use case: With big data, companies have access to large amounts of granular, detailed information. Manufacturing companies can use equipment details like manufacturer and condition, number of product runs and types of loads, and maintenance records to create a maintenance scoring system or the likelihood of equipment needing maintenance. In this use case, Coopers Brewery used big data to reduce waste. Coopers Brewery recognized that beer wastage occurs at every stage of production in the industry — including filtration, the transfer of beer between vessels, and when beer is packaged. And when any product is lost, so is the opportunity for generating revenue. To track how much beer it was losing and where, the company used big data to identify the root causes of wastage and find solutions. In the fermentation process, for example, they found that one particular machine simply performed better than the others. Addressing the root causes of beer loss helped the company save 70% in potential revenue loss due to wastage, while continuously improving operations. In an industry where decisions have historically been made based on experience and gut feel, big data helped Coopers Brewery make data-driven decisions.
Better big data tools and solutions
Big data allows you to be more flexible, agile, and proactive in identifying trends and making data-driven decisions. You can ask new questions of old data. Modern analytics tools, like Tableau, allow faster speed to insight. With big data, you can do old tasks faster and complete analysis that wasn’t feasible before. Sellers can now map the entire customer journey, observing all channels that customers use to interact with the company, including various web touch points, print, brick-and-mortar visits, and social media. Connecting touch points for a 360-degree view has only been possible in the last few years. These kinds of projects are feasible with big data solutions like Hadoop, SQL, and Tableau. Learn more about Tableau and big data in our whitepaper.