From Succeeding with AI by Veljko Krunic

Focused human involvement and attention makes or breaks an AI project. This article discusses the first questions you should ask in the context of every AI project, as well as the most important ideas to keep in mind during any AI project.

Take 37% off Succeeding with AI by entering slkrunic into the discount code box at checkout at manning.com.

Start by understanding the possible business actions

You don’t make money by knowing the answer to a business question—you make money when you take action, and that action is constrained by what you can and can’t do in the physical world (for example, the number of the possible management decisions). A business action can require approval from your boss(es), the creation of external partnerships, and getting the whole team to buy in on that decision—this is the real world we’re talking about.

TIP  The number of good, effective actions you can take to affect the physical world is relatively small. Many analyses can be performed which yield results that aren’t actionable.

The limits to business actions you can take many forms. They may be limits of knowledge and know-how in your organization. Or the limits of your budget. They may be imposed by the organization’s internal politics and what you’re able to rally people around. Or they can be the result of which battles you’re willing to fight. Whatever has summoned them into being, we’re stuck with them now.

You can spend a lot of time on analysis, get some results, and then say, “Well, there’s nothing I can do to change this.” The time and money spent on such analysis is wasted, and it’s a preventable waste.

TIP  Don’t ask a question if you can’t imagine what you’d do with the answer. You should start an AI project by asking, “What actions can I take, and what analysis do I need to do to inform those actions?”

Once you know the business actions you can take, you should use these actions to drive the analysis, not vice versa. Figure 1 illustrates the relationship between the business actions you can take and the possible analyses you can perform.


Figure 1 You can perform many more analyses than there are business actions you can take. Don’t spend a ton of time and money on an analysis to figure out it was never an actionable one. (Earth image is from [28].)


The data always has something in it, and it’s possible to conduct hours of analysis on only a few hundred numbers.[1] Often, you’ll find some interesting properties in even a small dataset. The problem is that the large majority of those analyses fall into the category of this is interesting, but not relevant to the business. If you’re thinking about and starting with analysis, you can spend a lot of money and time on it and then not have any idea of how to execute on the results.

What about EDA?

A limited exception to the rule of “consider first what you’ll do with results before you perform analysis” is Exploratory Data Analysis (EDA), which helps your data scientists understand the structure and basic properties of your datasets. If an EDA effort is small (compared to the total project budget), it may be worth doing EDA  to better understand your datasets.
To claim to be exception to the rule, EDA and all preparatory work necessary for it (such is any cleaning of the data) must be a small effort of the larger project.
If the effort needed to perform EDA is instead a significant part of your project, you need to justify the collection of those datasets, decide on which datasets you’ll perform EDA on, and explain why you think there’s some value in looking at that data.

In every AI project, the two most important ideas to remember are:

  1. Action is where you make profit; analysis without action is cost. You make money when some action is performed in the real world, not when some analysis is completed. Analysis can be an enabler of making profits, but from an accounting perspective, it’s more of a cost center. Analysis is worth investing in when it can inform you to take better actions.
  2. To succeed, focus on the whole system, not on its individual parts. Your customers will never see the individual parts of the system. For instance, they don’t care about the ML algorithms that you’re using. They care only about the result, and that result depends on how well the system works as a whole.

That’s all for now.

If you want to learn more about the book, you can check it out on our browser-based liveBook reader here and see this slide deck.

 

 

 

[1]Improving business with data isn’t new. A long history of projects that were done in the factory and business process improvement space prior to the rise of big data exist. Those projects were done on small datasets, but often used complex statistical analyses which required high technical proficiency with statistics and significant time to perform. Many of such projects fall under umbrella of Six Sigma – see [29] and [30] for details.