rw-book-cover

Metadata

Highlights

  • Getting leaders to change their actions based on data and models is one of our most significant strategic challenges. Nothing can be done at a tactical level to drive that kind of change. Data teams paddle in circles unless someone at the strategy level takes on the challenge. (View Highlight)
  • I teach AI strategy and product management because if you trace the root causes of many execution challenges, you’ll find the symptoms originate from gaps in one or both. Data must improve outcomes to deliver quantifiable value. Outcomes won’t improve unless data can drive changes. (View Highlight)
  • In change management, a leader decides to make a change, and the frameworks support efficient implementation. With this challenge, the data or model indicates that changing will improve outcomes. We need frameworks to get leaders to listen to data like they listen to experts and their own intuitions. (View Highlight)
  • The definition of insanity is doing the same thing repeatedly and expecting better results. Doing the same things with data doesn’t improve those activities’ value either. Data must inform change, so data teams focus on delivering data that indicates high-value changes. If leaders won’t act on data-driven recommendations or follow data that disagrees with them, value never materializes. (View Highlight)
  • We must prepare the business to use data or it never will. Most data teams are in an infinite loop. (View Highlight)
  • The highest-value data reveals opportunities for improvement and how that value can be realized. (View Highlight)
  • Leaders ignore data that doesn’t agree with them, and nothing changes. (View Highlight)
  • Business leaders talk about building an agile company but still can’t escape this loop. The inability to change stems from a lack of effective change management frameworks. It’s the root cause of many different business problems. Data teams are just the most recent victims. (View Highlight)
  • The last decade belonged to companies that took years to achieve profitability. The next decade belongs to startups that become profitable in months. (View Highlight)
  • Bigger businesses have too much inertia to compete with them. A startup can discover an opportunity on September 1st and deliver it by September 30th. In a large corporate environment, the idea may never get any visibility. Even if it does, opportunities spend months working through the approval process. Agility is just a word for businesses that won’t change. (View Highlight)
  • People make a very comfortable living solving change management problems, and data gives us one of the best paths in. Businesses know they must change to succeed with data and AI. Business leaders have been forced to confront change management. This is one of those rare convergences that creates a massive opportunity. (View Highlight)
  • The Bureau of Labor Statistics got into trouble by improving a model. What sounds like an excellent thing to data scientists can be perceived as a negative by users and stakeholders. Improving the model and increasing the frequency of model output reviews led to a dramatic downward revision in the total number of jobs created in the last 12 months. The BLS framed it as part of a standard annual comparison between its estimates and state unemployment insurance numbers, which is considered a more accurate benchmark. (View Highlight)
  • The public’s response wasn’t, ‘Good job. Thanks for correcting that mistake!’ It was, ‘The government is covering up the truth. The economy is terrible. We’re on the verge of a recession!’ The BLS didn’t explain what happened, so several analysts had to work it out on their own. It’s still speculation, but it makes sense. There hasn’t been a downward revision this big since 2009…The Great Recession. That was also a time when model assumptions failed to hold, and estimates were way off as a result. (View Highlight)
  • Business leaders expect data and models to work the same way software does. Enter 3 + 3 into a calculator app; the answer is always 6. Software is stable and deterministic. Data and models are stochastic. Deterministic systems are well-defined, and their dynamics or rules are well-understood. Outcomes are accurately predicted. (View Highlight)
  • Stochastic systems are partially defined and understood. Outcomes seem to have an inherent randomness that comes from the parts we don’t understand. Data changes over time. If I measure tidal surges for 6 months, I’ll get a good picture of how tides work. If there wasn’t a storm surge during that time, my understanding of tides won’t work for storm conditions. (View Highlight)
  • However, my dataset will be more complete after a storm hits, and my understanding of tides will improve. Models work until they don’t. We learn from model failures, and that leads to better models. In software, failures are unexpected defects. In data and models, failures are invaluable. (View Highlight)
  • Data and models answer much more complex questions than software, but model outputs are the most probable answer based on our data. As time goes on, we always get more data. When it’s the same as our current data, nothing improves. When data contradicts our models, we can retrain them to be more accurate. (View Highlight)
  • Businesses are also stochastic. People who do deterministic, well-defined work are considered low-skill. People who handle stochastic work are called knowledge workers and decision-makers. Software and digital systems have automated many of the business’s deterministic tasks. Now, we’re entering an era where data and models are trying to tackle stochastic tasks. Models and knowledge workers aren’t always right. Both learn from their mistakes and improve based on experience. (View Highlight)
  • If no one explains this to business leaders and implements frameworks to support stochastic systems, data and AI are set up to fail, along with the data team. Legacy processes and strategy frameworks are designed to support a deterministic business, but, in reality, the business has never been fully deterministic. (View Highlight)
  • If you look at long-term trend data, sales and discounting strategies can create a death spiral. From a short-term perspective, sales look like an easy fix when growth stalls. Getting the benefits of sales and discounting without falling into the death spiral requires data from across the business and complex pricing models. (View Highlight)
  • I have worked with several retail clients, and the models are never enough to prevent the discount death spiral. Leaders must be willing to change their minds and act on data that indicates a new discounting strategy. That’s a strategic decision-making problem that can’t be solved during the meetings when decisions are made. After explaining the difference between software and AI, we must implement frameworks to support the stochastic business. (View Highlight)
  • If you know business leaders don’t trust or use data to make decisions, that can’t be our starting point or primary supporting evidence. Storytelling is a powerful persuasion tool we can use to introduce everything else. We must Meet the Business Where It Is, not where we think it should be. (View Highlight)
  • We want to train CEOs to ask the right questions because we can answer them with data and models. A stochastic question doesn’t have a single answer. It has a most likely answer based on what we know, but there are also less likely alternatives. Most models aren’t built to present all options in response to a question or all the risks associated with an option. (View Highlight)