10 Ways to Go From Unhealthy to Healthy Ad-Hoc Request Cycles

Metadata

• Author: count.co
• Full Title: 10 Ways to Go From Unhealthy to Healthy Ad-Hoc Request Cycles
• URL: https://count.co/blog/10-ways-to-go-from-a-toxic-to-a-healthy-ad-hoc-request-culture?ref=nla&utm_source=newsletter&utm_medium=email&utm_campaign=adhoc-a&utm_content=article

Highlights

• Ad-hoc requests have a bad reputation. At their worst, they are a data team’s nemesis - distracting the team from ‘valuable’ and ‘interesting’ projects, causing loads of rework, without having any idea how important any of these requests really are. (View Highlight)
• However, there is another world in which ad-hoc requests are not the monster we imagine. Where ad-hoc simply demonstrate a curiosity and a desire to use data to understand their own world. Where a ‘quick question’ could be an olive branch, inviting data teams into the business’s biggest challenges. (View Highlight)
• We’ve all experienced unhealthy ad-hoc cycles. This is when we’re asked for ‘a quick question’ in Slack that turns into a 3-day scavenger hunt for an insight no one can describe. It’s when you finally get to an endpoint, think you’ve helped the business find something valuable, an insight that might change the course of how that team works, or maybe just improve their metrics that quarter, only to find out they decided to do what they were going to do anyways. It’s exasperating, draining, and fruitless. (View Highlight)
• healthy ad-hoc can feel oddly…good. When you get a request from the business, you can usually anticipate why they’re asking - what hypothesis they have that can crack the big problems you helped them identify. You are brought into their problem-solving, helping each other come up with new theories to test and lines of thinking to explore. Your time spent doing this is often more valuable than the time spent building another report or answering a vague data request. These are the days we love most in data. (View Highlight)
• The first step in getting a handle on your ad-hoc request problem is to track and measure it. Amanda Sjöstrand Henriksson, Analytics Manager at Insurello suggests creating a simple spreadsheet to track: • Requestor team & individual • Request details • Impact (1-5) • Estimated time • Actual time • Effort (1-5) • Follow-up questions & requests We, of all teams, know the importance of measuring what matters, so this is a fundamental step to getting a handle on your ad-hoc request problem. (View Highlight)
• With your auditable list from the step above, you can now begin to use it to make some tangible changes. Anzisca Van Rooyen, Senior Data Analyst at Teya advises:

  “Most of the time ad-hoc requests can be grouped. For example, once a week someone asks for a list of customer details - build a self service product. A dashboard that always needs debugging- find the route cause or save the code/response needed to debug.” Jerrie Kumalah, Analytics Engineer at Seat Geek echos this: Figuring out the themes that are emerging from these ongoing ad-hoc requests. They often follow a pattern which will help the team build better processes. If you’re looking to make major changes to your ad-hoc request process, this is where you can make the biggest difference by actioning the common sources of requests. (View Highlight)

• If you have been drowning in ad-hoc requests for long enough you are bound to be cynical about them and are likely looking for this article to teach you how to eliminate them. That may not be the best approach according to Amanda Sjöstrand Henriksson, Analytics Manager at Insurello:

  “[Ad-hoc requests] can give you the chance to work closely and in-the-moment with your stakeholders, to be there when things happen and work as a team rather than an outsider. Before you set off to ‘fix’ this problem, you have to remember what works about ad-hoc requests, too.” Will Sweet furthers this idea by encouraging us to see that ad-hoc requests are actually a great way of building trust with the business. “View each question as an opportunity to build more trust.” [1] Guillaume Ansanay-Alex reminds us that ad-hoc requests actually contribute to a company’s bottom line due to what he calls the cross-fertilization between business topics and teams: “Tech workers likely get requests from all around a business unit or even the whole business. Getting higher exposure to business contexts enable them to connect the dots between different requests. Like in, ‘We are talking about setting store pricing levels for various regions, but did you know that we worked on a multi-feature segmentation of stores for the product folks? Wouldn’t it be an even better approach?’” [2] In summary, banning ad-hoc requests entirely may eliminate the pain you feel, but it will also come at a significant cost and diminished impact on the larger impact. Our energy will be better spent focused on how to do them well. (View Highlight)

• A good first step, then, is being able to classify the ad-hoc requests we receive into different categories. Ryan Dolley, VP of Product Strategy at GoodData has defined two types of ad-hoc requests:

  “High-value ad-hoc questions deliver insights that materially improve business outcomes. They may require new ways of thinking supported by new data pipelines or metrics. They lead people to take action. High presentation-purpose fit leads to high-value ad-hoc questions.” “Low-value ad-hoc questions aren’t unimportant, but they don’t have a large impact on the business. These are often reconfigurations of existing knowledge to show a slightly different slice of the business. Think, ‘Can I see this metric by quarter?’ Low presentation-purpose fit leads to low-value ad-hoc questions.” [3] (View Highlight)

• Much like how product teams have to leave room to bug fix, data teams should leave room to answer ad-hoc requests. Dinda Calista, Analytics Lead at Cleo tells us that there analysts there are expected to manage all of their tasks, including ad-hoc requests, in their squad backlog. This effectively puts ad-hoc requests and scheduled projects on equal footing, rather than secretive things analysts do in their spare time that cause their core work to be late. The advantages here are:
  1. The PM and the squad have oversight over all of the work an analyst is doing, rather than ad-hoc requests being hidden. This also means the analyst and the requester have to justify why they think those ad-hoc requests are worth their time.
  2. The PM can also step in and make sure an analyst isn’t spending too much time on an ad-hoc request that doesn’t seem valuable enough or is dragging on for too long. (View Highlight)
• Getting the right information for each ad-hoc request is crucial to deciding what to do about it. We received many suggestions about what the right questions to ask are, and what your questions should be achieving, but here are the three highlights that emerged: [6.1] Why now? Ad-hoc requests are often stressful because they come with an implicit urgency. Everything feels like it was needed yesterday. It is therefore an imperative question to ask not just why this request is needed, but why is this needed now. [6.2] Make sure your questions help you find alignment. Will Sweet reminds us that asking questions is not purely for gathering requirements, but is also a crucial step in establishing alignment.

  “Finding common ground builds empathy, and there’s no better way to communicate with someone then to align your goals and theirs.” [1] Making sure your questions help reveal not just the requestor’s motivations, but also your own, is fundamental to establishing alignment, and thus, trust. [6.3] Make it about actions Cassie Kozyrkov, Chief Decision Scientist at Google, encourages an action-orientated approach when it comes to acting on an ad-hoc request (or indeed any data request). This emphasis helps remove emotion and bias from a request and ensures that you and your team are only working on things that will affect the actions the business will or will not take. “If you don’t have a default [action], you don’t need fancy statistics.” [5] (View Highlight)

• Asking the right questions is only helpful if you can use them to act, and sometimes the action you will need to take is to say No to an ad-hoc request. Will Sweet offers several tips on how to say ‘No’ to your business, most of which do not actually involve saying ‘No’, but maybe ‘not right now’ or ‘not quite in that way.’ Regardless, applying friction to these requests is essential in ensuring your team is working on the right requests at the right time. [1] Amanda Sjöstrand Henriksson echos this sentiment, however focusing on the need for data teams to lead business requestors into the right approach to their problem.

  “Your stakeholders probably do not know ways of working in an analytics team and what is an ad hoc request or not. It is the analysts job to guide the stakeholder - they only come to you with a problem. And if you see it fitting the ad hoc framework or if it rather should be a big project - that is your decision and responsibility. So guide the stakeholders!” (View Highlight)

• Once you have a clear understanding of what is required, the urgency, and the outcomes of the request, it is important to choose the right approach. Ryan Dolley refers to this step as ‘presentation-purpose fit’, which means are you choosing the best tool for what you need to communicate.

  “If you fail to build a delightful front end, all of your backend work may be wasted.” [3] Phuong Nguyen, Growth, BI at ZaloPay furthers this by reminding us that: • sharing dashboards can be a good way to give operational oversight, or let people filter and explore some data at a superficial level • sharing queries is perfect for the user who wants to explore on their own, and wants to learn more about data at the same time • receiving ‘why’ questions usually requires a deeper and more meaningful analysis beyond both of the above methods, but these are often the most valuable requests we receive. [6] (View Highlight)

• Sharing the results of an ad-hoc request is about more than the tool and output you choose; it is also about how you communicate what you’ve found. Will Sweet breaks down the three components of effective communication [1]:
  1. The header, or TL;DR: this is a very short headline of what you’ve found
  2. The story: this is the context and background into how you arrived at that headline summary
  3. The methodology: this is your story of how you arrived at this outcome Depending on the request, you may need to emphasize some of these more than others. Big decisions, or actions, for example, may require more methodology as they require more trust to be established before moving forward. (View Highlight)