From Succeeding with AI by Veljko Krunic
When running an AI team or project there are important pitfalls to be avoided. This article discusses some of the most common ones.
Pitfalls to Avoid
When running an AI team, there are some popular pitfalls that you should avoid. Some of the most important ones are as follows:
- Not communicating with the organizational actors who own the React part of the Sense/Analyze/React loop. Or, even worse, not working with them at all until your AI project is well on its way.
- Transplanting use cases (and metrics) from other projects or organizations.
- Running fashionable AI projects which are likely to grab headlines.
- Believing that you can buy a tool, any tool, which will give you a sustainable advantage.
- Hoping that throwing random analysis at your data will produce results.
- Selecting which project to run based on a “gut feeling” instead of the results of analysis.
This article discusses each one of these pitfalls in more detail.
Failing to build a relationship with the business team
When using AI as a decision support system, it’s never enough to deliver a good analysis; you need to execute well on the specific business actions recommended by an AI-powered analysis. This means that executive attention must home in on the link between the analytical result and the business action. This section highlights why the AI team must build good relationships with the department of your organization which takes business actions, based on your AI analysis.
Analytical results can be misinterpreted by a non-specialist. A classic problem is that non-specialists don’t understand the limits of the analysis and when the assumptions basic to the analysis are violated. An example of that problem was shown in section 3.3.1, when the research question and business question were misaligned.
I’ve personally witnessed several organizations hand off an analysis report to separate business teams. These business teams proceeded to take business actions without the input of data scientists. This is always a mistake.
If you’re the leader of an analytical project, your job isn’t done when you deliver the results. Your job is done when the analyst’s prescription is successfully implemented. You must build good working relationships with the departments that implement the analysis. If the analysis has prescribed a particular business action, don’t underestimate the need for you to help with and follow through with its execution.
People are often tempted to copy what worked for the people and organizations that surround them. As a result, you see what I call transplant projects. Here, an enterprise decides to form an AI team, and embarks on some AI project they’ve heard other organizations similar to theirs performed. This section explains why transplants are a bad idea.
Examples of transplant projects abound. Some examples are projects like “let’s have our own recommendation engine” or “let’s do sentiment analysis of customer feedback.” Sometimes these projects make sense in the context of the business, but all too often they’re vanilla use cases that you heard about from someone else and didn’t analyze in the context of your own business.
Instead of blindly adapting a project that worked well for someone else, consider it to be one of many possible AI projects.
Trying moonshots without the rockets
Many of the world’s largest technology companies have made fortunes based on the use of data. In the core, companies such as Google, Microsoft, and Baidu are heavily dependent on AI for their success. They’ve significant research capabilities and have a vested interest in ensuring that they won’t miss the train of important AI advancements. This section explains why your organization shouldn’t blindly follow those companies.
Although the previous logic applies to businesses such as Google, Baidu, or Microsoft, there’s an unfortunate tendency for many enterprises to emulate these companies without understanding the rationale behind their actions. Yes, the biggest players make significant money with their AI efforts. They also invest a lot in AI research. Before you start emulating their AI research efforts, ask yourself, “Are you in the same business?”
If your company was to invent something important for strong AI/AGI , would you know how to monetize it? Suppose you’re a large brick and mortar retailer. Could you take full advantage of that discovery? Probably not – the retailer’s business is different than Google’s.
Almost certainly, your company would better benefit from AI technology when used to solve your own concrete business problems. This means that instead of teams populated by the smartest researchers and processes oriented toward the acquisition of new AI knowledge, your organization needs people who know how to make money in your business domain with existing AI technologies.
Don’t emulate organizations richer than yours without first understanding how you’d exploit success. For most organizations, the road to success isn’t found in advancing the frontiers of AI knowledge, but in knowing how to tie AI results into their business. You need a data science team focused on applications, not research. That doesn’t mean you shouldn’t hire bright PhDs, but that the leadership of your AI teams must primarily be experts in applying AI to the task of making money.
It is about using advanced tools to look at the sea of data
Another common pitfall is the belief that you can buy an AI or big data tool which makes it trivial to look at your data, find insights, and then monetize the insights found. Some organizations adopting AI might even take the attitude that the main focus of early AI efforts should be on finding the right tools. This section explains why this is a pitfall to avoid.
In most business verticals, it isn’t trivial at all to monetize AI. And although many tools can help you get there, it’s unlikely that these tools can solve monetization problems for you. Even if there are tools that let you monetize by installing and running them, what you’re dealing with is a commoditized use case. Heck, someone already has a product that does it!
A salesman might advise you to “Build a large data lake and unleash your data scientists on it; there has to be something in all that data.” You might even have been given an example of the unexpected insights that only analytics on a big dataset can provide, but those situations are rare and unpredictable. Don’t count on the tooth fairy. Don’t start with the Analyze part of the Sense/Analyze/React loop. In our framework, always start with the React part.
Using your gut feeling instead of CLUE
Often a decision about running an AI project is made in haphazard way, as little more than a technical idea that excites the team. Running AI project primarily because you want experience with the underlying technology is the tech equivalent of buying a sports car. This section explains why “following your gut” may result in poor business results.
Be extremely skeptical about counting on intuition to select which AI project to run first. Section 3.2 showed you the steps necessary to correctly determine which is the best project to run. When selecting a first project, there are too many moving parts to consider for gut feeling to provide the right answer. You need to verify that the project is actionable, technically possible, and business valuable. You need to know its cost, as well as the outline and difficulty of the proposed technical solution. It’s highly unlikely that you can assess all those attributes of a proposed AI project by thinking about the problem for a minute or two.
The biggest cause of failure of AI projects today might be technical, but even among technically successful projects, there are far too many that aren’t even used by the businesses that paid for those. Those AI projects shouldn’t have been started at all and were usually started because a gut feeling about their value was wrong.
That’s all for now.