Data & AI health is hard to define. Recently, it occurred to me that its essence can be distilled with three questions:
- Plumbing: What’s the state of your data engineering lifecycles?
- Decisions: How do you use descriptive, predictive, and causal modelling to support decisions?
- Automation: How do you use AI to automate processes?
These questions help identify gaps and opportunities. While each question focuses on the present state, it’s natural to follow up with plans for a brighter future.
In practice, you would go deep on each area. Each question is a door that leads to a corridor with many more doors.
Amateurs versus professionals
If you’ve ever worked with data, you’d have a sense of what amateur and professional answers to the above questions may look like. In practice, answers are multifaceted and fall on a continuum. But here are some simplified examples from each end of the continuum:
| Amateur | Professional | |
|---|---|---|
| Plumbing | Rudimentary pipelines, manually-populated spreadsheets | All necessary data is trustworthy and available on tap |
| Decisions | Relying on one-off charts and models, along with the intuition of HiPPOs (highest-paid persons’ opinions) | Relying on relevant data and modelling efforts that are proportional to the gravity of each decision |
| Automation | Superficial use of off-the-shelf tools | Deep, mindful integration of tech to replace manual work where it delivers the most value |
Going down the rabbit hole
The three areas pretty much define my career, but there is always much more to learn. The main message of this post is that little has changed since Harrington Emerson uttered these words in 1911:
As to methods, there may be a million and then some, but principles are few. The person who grasps principles can successfully select their own methods. The person who tries methods, ignoring principles, is sure to have trouble.
(OK, one thing did change – Emerson used man rather than person, but I fixed it for him.)
You can explore further with these posts:
- Plumbing: Fully understanding the data engineering lifecycle is more important than mastering a single tool.
- Decisions: According to my 2018 definition, this is what data science is all about. There’s endless depth to building descriptive, predictive, and causal models. But the key to rising above tool hype is understanding the why of data science, which is to support decisions.
- Automation: The term AI is around peak hype right now. This makes it easy for cynics to dismiss the over-excited claims of AI proponents. Avoid cynicism – simply think of AI as automation and understand that relentless but mindful automation is key to success in our world.
More questions to probe the Data-to-AI health of startups
This post is a slight detour from the series on my Data-to-AI Health Check for Startups. I figured it’s a valuable detour since I now see the triad of Plumbing, Decisions, and Automation as the essence of Data & AI health for any organisation.
Previous posts in the series:
- Assessing a startup’s data-to-AI health: Overview and motivation
- Business questions to ask before taking a startup data role
- Probing the People aspects of an early-stage startup
- Question startup culture before accepting a data-to-AI role
You can download a guide containing all the questions as a PDF. I’m still planning to cover Processes & Project Management next – hopefully I won’t get detoured again. Feedback is always welcome!
Public comments are closed, but I love hearing from readers. Feel free to contact me with your thoughts.