Primary Submitter:
Lars Böhnke, lars@candis.io
Domain:
Prospect vetting for SaaS-developer / provider
Generic description of sponsoring organization or customer:
Series B startup located in Berlin, Germany, providing invoice management software
Cognitive Task Analysis Method(s):
A Goal-Directed Task Analysis (GDTA) & Knowledge Audit (KA) was performed with business development representatives (BDR’s; N=2) of the vetting process, followed by a GDTA & KA, as well as a Critical Decision Method (CDM) with accounting executives (AEs; N=3) working at different stages along the three-staged vetting sales funnel. We also employed a signal-noise and cue-profile analysis for heuristic construction.
The goal of the BDR-interviews was to gain a first overview of the vetting process (goals, available actions, persons involved, decision process (e.g., individual vs. group-decision making), etc., and to capture the overall flow of prospects through this process. It was also used to identify diagnostic information (cues) used in the organization for vetting potential prospects at the different stages of the funnel (AE interviews).
We furthermore identified the information / data available in the organization and the requirements for evaluating process performance in a Signal Detection Analysis framework (i.e., hit, correct rejection, false alarm, false negative). Lastly, we identified suitable interview candidates at the three levels of the prospect-vettingfunnel for further interviews. The goal of the interviews with experienced AEs was to validate the process description and to validate / enrich the set of cues used by experienced sales employees for prospect-vetting, as well as get an assessment of their diagnostic validity (sensitivity & specificity). Initially, this was done by interviewees having to evaluate the relative strength (“clearly dominates” vs. “is weighed against”) of the cues, then by providing a numeric estimate (percentage successful vs. unsuccessful prospects exhibiting a cue). Lastly, interviewees were presented with prospect cases with permuted cue-profiles as asked to assess the proportion of successful vs. unsuccessful prospects. This allowed for an evaluation of the diagnostic validity of individual cues as well as their combination into a full set of cues. This analysis also served to assess the base rate of suitable vs unsuitable prospects at different levels of the funnel.
The CDM in particular was used to identify possible sources of error, identify work-arounds from the formal vetting process, as well as boundary conditions / special circumstances that could trigger deviations from it.
Number of Participants:
N=5. Two BDRs and three AEs working at different levels of the prospect evaluation funnel.
Duration:
One calendar month, mid-May to mid-June 2022
Application(s):
KA and GDTA resulted in a heuristic process of the standard procedure of prospect-vetting in the organization. The heuristic was used to train BDRs to disqualify prospects as earliest as possible in the vetting process. The identified key attributes by which prospects are currently vetted, and along which they should be vetted in the future were used to improve demo booking forms on the website to qualify demo bookings. The CDM was used to probe boundary conditions under which deviations from the standard procedure will occur / are desirable. The CTA for Heuristic Construction provided information about the relative diagnostic validity of individual cues and, for purposes of cross-validation, the assessments of full cue profiles as well as base-rates for the different levels of the vetting-funnel.
Demonstration of value:
Evidence of value
During January and June 2022, 53 opportunities were disqualified with the reason “no prospect
fit.” We were able to reduce this number to 17 during July and December 2022. The additional
capacity of BDRs and AEs was used to invest more time into opportunities from high-fit prospects.
Customer-provided perspective
“By codifying the intuitive knowledge of our account executives about our market into formal training material, we were finally able to better train our existing BDRs which often felt lost because they had no process for orientation for their prospecting meetings. The training material also sped up the onboarding of new sales employees. In addition, we clearly communicated factors that make a prospect not a fit to the marketing team to disqualify prospects as early as possible on our website.” – Head of Growth
Citations
Used in the case study
Crandall, B., Klein, G. A., & Hoffman, R. R. (2006). Working minds: A practitioner’s guide to
cognitive task analysis. MIT Press.
Keller, N., U. Czienskowski & M.A. Feufel (2014): Tying up loose ends: a method for constructing and evaluating decision aids that meet blunt and sharp-end goals, Ergonomics, DOI: 10.1080/00140139.2014.917204.
Keller, N. (2011), “Bridging the Gap between Strategic Trade-Offs and Individual Soldier’s Cognition: fast and frugal trees and force protection in counter-insurgency operations”, Proceedings of the 10th International Conference on Naturalistic Decision Making, May 31st to June 3rd, 2011, Orlando FL, USA. M. Fiore & M. Harper (Eds.). Orlando, FL: University of Central Florida.
Keller, N., Cokely, E.T., Katsikopoulos, K.V., & Wegwarth, O. (2010), “Naturalistic Heuristics for Decision Making”, Journal of Cognitive Engineering and Decision Making, 4:3, pp. 256-274.
