To paraphrase Raymond Chandler, too many projects deliver department store data: The most of everything but the best of nothing.

Enterprise AI and analytics developers must avoid the mistake of underserving people by overengineering solutions. Designers and decision makers need straightforward tools to make them better, to save time and facilitate their best work. They don’t care about the latest dashboards or algorithms unless they create value. Oh, and they don’t want to hear about how AI might take their jobs.

“Simply put, more advanced techniques give you a lot more rope to hang yourself with.” -#ODSC

No model shaming! In business environments, complex tools can be distractions, slowing things down while missing the mark. The Data Science Community (#ODSC) gives a wonderful example in Save a Neural Net, Use a Linear Model. “The temptation to go for gold and use state-of-the-art techniques like neural nets is ever-present among machine learning rookies, and it is a constant source of consternation among managers and more experienced data scientists.”

Despite what you read in the press, professionals routinely create linear models free of embarrassment. To keep your analytics or data science team funded, it’s wise to provide good-enough tools that are useful to decision makers.

These four examples of interesting tech illustrate why we should avoid the tendency to offer the most, not the best.

Example #1: Principle components analysis vs. writing talent.
Sophisticated AI can drive cars and recognize faces, but it can’t replace creative talent. Nevertheless, the New York Times Book Review showcased a rigorous analysis of Jane Austen novels, exploring whether literary genius can be graphed. Evidently Austen used more abstract words early on, and more physical descriptions post 1780. She referred to women more frequently than most British fiction. But didn’t we sort of know that?

The graph is beautiful, but does it (as the headline claims) truly “explain why Jane Austen’s work survives and thrives”? One contributor says “To reduce words to data points is not to diminish the power of choosing the right ones and putting them in the right order…. But a discipline like statistics can offer a new and amusing — even instructive — look at something as intuitive and artistic as a novel.”

Amusing data viz won’t save your job. Your business unit leader expects something more actionable than beautiful graphs. The Book Review acknowledges Virginia Woolf’s observation that “Of all great writers, [Austen] is the most difficult to catch in the act of greatness.” Amen to that.

“Every block of stone has a statue inside it and it is the task of the sculptor to discover it.” -Michelangelo

Example #2: Special effects vs. structured scriptwriting.
Movie scripts are quite formulaic, applied with tremendous effect by screenwriters like Nora Ephron and William Goldman. If sophisticated AI made a movie, you’d likely get something formulaic like Transformers: Safe, shiny FX with little emotional resonance. But a talented writer can follow a script structure, shape a specific point of view, and produce Bumblebee. If they’re really good they create classics like The Princess Bride or Galaxy Quest.

Example #3: Testing hypotheses vs. forming them.
The reliably good Cassie Kozyrkov @quaesita wrote an enlightening post about decision-making and hypothesis testing. Key point: “Statistics classes teach you to test hypotheses, not form them. They tend to be pre-made for you on those exams.” That’s the easy part. Learning when and how to form good hypotheses is much harder, and much less likely to be automated.

Example #4: Reporting vs. analyzing.
Several publishers have developed AI to generate story ideas for contributors, including relevant headlines and images. Forbes offers Bertie, suggesting rough drafts to reporters. This is great because Bertie doesn’t attempt to build a department store, it simply recommends snippets to capable humans. (Sometimes AI tools do write short news bits, such as minor league baseball recaps, but these follow a repeatable structure to report stats, not generate analysis.)

Tracy Allison Altman leads The Deciding Project. (Imagine people & AI deciding, together. Now throw art and comedy into the mix: What could go wrong?)

Posted by Tracy Allison Altman on 16 January 2019.