Overcoming hurdles in end-to-end AI project design

Editor’s note: This article originally appeared in Forbes.

According to a recent study by 451 Research, part of S&P Global Market Intelligence, “more than 90% of organizations that have adopted AI began development on their first AI project within the past five years.” Though nascent, AI-enabled solutions are on the rise all around us. However, many of these initiatives still aren’t meeting expectations—if they even make it to deployment.

To succeed, leaders should select and manage AI projects with a thoughtful strategy driven by clear expectations, alignment to business goals, and iteration. Let’s look at common hurdles organizations face when designing successful end-to-end AI projects, and how to overcome them.

Managing expectations for AI-enabled solutions

A lot of today’s failing AI projects today are reminiscent of enterprise software projects in the nineties, where development projects were going off the rails as teams held high hopes that new technologies would fix their problems. Both then and now, a major pitfall is having inflated expectations of what your solution can actually solve.

It’s dangerous to assume that by collecting enough data, everything will suddenly be transparent; that you can predict customer behaviors or make perfect recommendations to anticipate their needs. Unfortunately, the world is far less predictable than people want it to be. While helpful patterns do emerge, not all events are causal or even correlated—many things happen that just generate noise.

At the same time, many organizations see their peers implementing AI solutions and feel the pressure to keep up. Investing in AI just to “keep up with the Joneses” can backfire if you don’t understand what’s driving your peers’ successes and whether or not it will work for your own organization. Often, companies with a leg up in their AI projects have data strategies and business processes in place that enable them to collect and take advantage of the right kinds of data for AI.

Ultimately, managing expectations for AI projects starts with the ability to articulate which of your problems can really be solved with AI.

Choosing the right kinds of AI projects for your problems

Is your AI strategy aligned with your business goals? Project selection is probably the single greatest challenge that organizations face with their AI initiatives. It’s important to really understand the question you’re trying to answer, how (and if) answering that question will deliver improved business outcomes, and whether or not the resources you have can answer it successfully and efficiently.

Let’s say you want to use a predictive model to determine when and what kind of discount to offer a customer. Bring in the data science team! But this is actually very challenging to approach as a predictive model problem. First of all, it’s hard to know whether or not your customer would buy the product without the discount. And gathering the necessary data with enough statistical rigor to produce a useful model would likely involve some processes that feel unnatural to the business—like randomizing which customers get discounts or which sales reps can give discounts. That adds a lot of complexity to the situation.

A better way to approach this problem with AI might be to explore simulation models of the customer behavior that you expect under different discounting regimes. Rather than torturing the system to arrive at a precise forecast, simulation and scenario planning can help people uncover which variables are sensitive to one another when making business decisions. Ask yourself: what customer response would we need for this discount to make sense? This kind of exercise in exploring potential outcomes is much more effective and certainly much easier than building up a complex data science experiment. 

Setting your teams up for success

Understanding what your data was collected and curated for, how it's been used in the past, and how it's going to be used in the future is critical for doing any kind of AI activity on the data. It’s important to train a model on data that’s complete and that represents what’s available in the real world at the moment you’re making the intervention. For example, if you have multiple stages in your deal pipeline and you want to predict the likelihood of a deal closing during stage five, you can’t then run the model on deals in stages three or four and expect useful results.

Data scientists often have a gap in understanding the nuances of what data represents and how it’s generated. What human and technological processes play a part in creating the data, and what exactly does the data mean in the context of your business? This is where analysts and business users who are close to the data—and the problems you’re trying to solve with it—are incredibly valuable. We like to think of AI as a team sport because success requires business context in addition to a baseline of data and model literacy.

Finally, there are human-centric aspects of project success that organizations can overlook if they’re too focused on the data or the technology. Often, AI can make a prediction but it’s up to someone to decide how to turn that into a recommended action. Is the suggestion useful in providing a clear action, and one that people will be willing to follow? Are you creating an environment where these suggestions will be received effectively?

Predicting something is only sometimes useful. Are you willing to adjust prices, volumes of product, or staffing, or even change your product line? What level of change management is needed so that people will embrace the new solution and evolve their established behaviors and processes? Trust comes from a pattern of consistent behavior and a willingness to continue to educate the business; if you’re going to radically impact how people do their jobs, they need to be on board with it.

Starting small and iterating

Let’s close on some guidance based on what we’ve seen while working with customers.

Oftentimes the best first AI project is the one that will be the easiest to operationalize and get into production with the least complex change management. Try to build something that delivers value as quickly as possible, even if that's a very small incremental improvement. And keep your customers, business users, and stakeholders as close to the development process as possible. Aim to create an environment of good feedback—both in the sense of collecting more data to iteratively improve the model, and input from stakeholders to improve the project and its outcomes.

With AI, there will always be edge cases where the solution is a miss. But it’s better to find solutions that work for the majority of your customers or employees, instead of developing a really flashy proof of concept that only works for a few bespoke use cases. At the end of the day, AI should reduce friction and make it easier for people to do their jobs and make informed decisions.

Want more? Check out our conversation about AI projects on LinkedIn, or learn more about Tableau AI analytics.