|
Almost Timely News: 🗞️ How to Do Great Focus Groups with RPGs and AI (2026-02-08) :: View in Browser
 |
The Big Plug
Two new things to try out this week:
1. Got a stuck AI project? Try out Katie’s new, free AI Readiness Assessment tool. A simple quiz to help predict project success.
2. Wonder how your website is seen by AI? Try my new, free AI View tool (limited to 10 URLs per day while it’s in beta). It looks at your site and tells you what an AI crawler likely sees.
Content Authenticity Statement
100% of this week’s newsletter content was originated by me, the human. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future.
Watch This Newsletter On YouTube 📺
Click here for the video 📺 version of this newsletter on YouTube »
Click here for an MP3 audio 🎧 only version »
What’s On My Mind: How to Do Great Focus Groups with RPGs and AI
This week, let’s look at a very specific use of generative AI, especially for marketing, sales, brand alignment and more. We’re going to make use of some gaming technology and show you how to ask your ideal customer for just about anything.
Part 0: TLDR
Use RPG character cards for business-focused role play with AI. They’re highly efficient and very effective.
Part 1: Glossary
Since RPGs are a bit of a niche, let’s do some table setting, almost literally.
RPG: Role playing game. This is any turn-based role playing game where you play characters on an adventure of some kind, either in the physical world or electronically. The most popular and well known is Dungeons & Dragons, though there are literally thousands of different RPGs. This also encapsulates games like World of Warcraft.
Character cards: Character cards are condensed summaries of a character’s traits and attributes, things like strength, intellect, agility, etc. Character cards are important because they condense a tremendous amount of information in a very tight space.
ICP: Ideal Customer Profile, a research report of who your ideal customer is - who they are, how they think, what makes them tick. Great ICPs can be very large, very thorough documents.
Tokens: The unit of text that AI uses, roughly 3/4 of a word depending on the system. Tokens are mathematical representations of things like words, pixels, or other small units of data. Because tokens are smaller than words, an AI model that has a context window of 200,000 tokens can remember about 150,000 words.
Context window: The short term working memory of an AI. Context windows are measured in tokens, and when they’re full, AI performance degrades or sometimes just outright fails. This is why we compress ICPs into character cards — to leave room for the actual focus group conversation.
YAML: A scripting/data serialization language of sorts, YAML stands for Yet Another Markup Language or YAML Ain’t a Markup Language, depending on who you ask. YAML is a well known way to format data that is friendly to both humans and machines.
Local maximum: in statistics, a local maximum is the highest value of a small region of the data, even when it’s not the highest value of the complete dataset.
Part 2: The Most Boring Role-Playing Game
If you’ve ever played a game like Dungeons and Dragons, there’s typically a game master (often called a dungeon master or DM) and then the players playing their characters. Think of it like a conference panel, but something you’d actually want to do (I despise panels); the DM is the moderator and the players are the panelists. Players take turns making their way through a narrative or story, playing characters like warriors, priests, rogues, and many others.
Each character has attributes and a personality, often based on their class, or type of character. Warriors and paladins wear heavy plate armor and absorb damage, whereas mages wear cloth armor and deal damage at a distance (which means if an enemy gets too close, a mage can easily be killed without the protection of a better armored class).
Now, let’s turn our attention to the venerable focus group, a collection of potential or actual customers who are sat in a room with cheap bagels and crap coffee and asked questions by a focus group moderator about their buying habits and preferences, so that a company can make better decisions about marketing, product market fit, strategy, etc.
Focus groups are also notoriously expensive and vary wildly depending on the skill of the moderator. RPGs are the same, by the way - a bad DM makes for a bad game.
A focus group is the most boring role playing game you can play. It’s important, but… usually not exciting. And it’s often templated, with the moderator asking questions like, “If you were to buy from a value added reseller in the IT space, which vendors come to mind?” A good focus group surfaces new insights, new ideas, or changes your thinking based on how the customer actually thinks and what they say. A bad focus group reinforces your existing biases, just like any bad market research.
The templated part is important. 11 years ago I said on stage, back in 2015, “If you do it with a template today, a machine does it without you tomorrow.” That’s more true than ever because…
Part 3: Enter AI!
Generative AI - specifically large language models - are language engines. They’re great at language, but one of the challenges with them is that they’re also probability engines. That means they return the highest probability result based on prompts they are given. If you ask your favorite LLM to “write a blog post about B2B marketing”, you’re going to get absolute slop, garbage, uncreative, unoriginal stuff. Why? Because things that are creative and insightful are by definition low probability.
An engine that returns high probabilities by default is going to struggle generating something low probability. Which brings us to a very important math concept, the concept of the local maximum. Yes, AI is a probability engines that returns the highest probability things from its internal database. But it’s not global, meaning that it doesn’t respond with the highest overall probabilities - it returns the highest probabilities from a selected set of information.
We are the ones to decide that selected set.
Here’s a concrete example. If I write:
“Write a blog post about B2B marketing”
I will get a pile of slop.
If I write:
“Write me a blog post about B2B marketing for a boutique analytics, data science, and AI consultancy in the Boston metro area trying to attract buyers who work for mid market industrial ball bearing manufacturing firms”
I’m going to get something that’s less sloppy. Why? Because all those extra words constrain what the probability engines can return. It HAS to consider things like the words “industri