From AI-enabled to AI-driven

In October I gave a talk at ShopQuest in Bangalore about where I think support is headed. It was our second year sponsoring as STOQ and my first time going myself. I was honestly a bit blown away by how many people showed up, from so many parts of the world.

The short version of the talk: right now most teams are AI-enabled. People answer merchants and use AI to do it faster. I think in the next year or so that flips, and AI does the answering while people check its work. I’ve been calling that AI-driven.

I could be wrong about some of this, and I said so on stage too. Below is roughly what I said, with a bit more detail than I had time for.

Some background

STOQ is a preorder and back in stock app for Shopify. Kartik and I started it in 2022, and we’re pretty much dead focused on it being the best preorder app on the platform. I wrote more about how we got here in 2,999 Conversations.

Support has always been a big deal for us. Merchants mention our team by name in reviews all the time, which still makes my day whenever I read one. In the early days it was just Kartik and me answering chats. A few years ago we were five people, and now we’re 20.

What is support, really?

I started by asking the room a question. When you help a merchant, are you helping someone who’s stuck, or helping someone get something done? Or both?

Most people put their hands up for both, which I think is right. But we usually only think about the first one. Something broke, I’m stuck, so I reach out. That’s how support works at Amazon or your bank.

What we see with merchants is different. Most of them just don’t have time. They want the app set up so they can go back to running their store. When we looked at what merchants ask Quinn, our AI agent, about two out of three questions were some version of “how do I set this up?” Only about one in five was about something being broken.

The way I explained it in the talk was the buddy in a video game. In a lot of games there’s always a little character who shows up before a hard part and tells you what to do, like grab the fire sword because this boss is weak to fire. That’s much closer to what merchants want from us than a help desk.

How we did support until now

When it was the two of us, we used Tawk. It’s a free live chat tool and it’s genuinely good. A merchant pinged us, we got a notification, and one of us jumped in.

Once we hired, Tawk got messy with more than one person answering, so in late 2023 we moved to Crisp. We still use Crisp across all our apps today and I really like it.

The volume just kept going up though. We went from basically zero conversations when we moved to Crisp, to over 4,000 in one month last Black Friday. That’s a crazy number for a team our size.

And even with a great team, it’s one person per chat. Merchants know a support chat might take 20 or 30 minutes, so a lot of them only reach out once they’re really stuck. I think plenty of merchants would have loved that buddy all along. We just couldn’t hire enough people to give everyone one.

Using AI to go faster

In 2024 and 2025 we did what most teams did and used AI to help our own people. We built a page where merchants could move a waitlist from one product to another without asking us. We built an internal tool that tells the team what probably went wrong with an order. And we built a dashboard called Engine Room that reads through conversations and shows us which ones need attention.

That helped a lot, but it was still people doing the support, just with better tools.

We also tried a bunch of the support bots out there. They’re fine products, but STOQ touches so many parts of Shopify (products, themes, checkout, inventory, orders) that none of them really fit how we work.

Building Quinn

So this year we built our own. It started as a small experiment. I was already paying Anthropic $200 a month, and I figured I might as well try.

The first version was called Q and could only answer questions from our docs. In June we renamed it Quinn, and in July we turned it on for every merchant. Conversations went from 61 a week to 266 pretty much straight away.

I’ll admit we fell for the deflection thing at first. Everyone I followed was talking about deflection rate, where the bot’s job is to keep conversations away from your team. One of the goals I wrote in our launch post for Quinn was literally “reduce our conversation volume.”

It took us a while to see that this was the wrong way to look at it. Merchants weren’t trying to get past Quinn to a person. They wanted to get things done, and Quinn could help with that.

When Quinn started doing things

At first Quinn only answered from our docs. Then we let it look at the merchant’s actual store. Then we let it make changes: it suggests what to change and the merchant clicks Apply.

That changed how merchants use the app. A lot of them now open it and just type something like “I have this product, can you put it on preorder for me?”

In August, Quinn had more conversations than our whole support team.

This was a bit of an existential moment for me. I’ve always thought of us as a small studio that builds a few Shopify apps and does them really well. It says that on our website. But at this point we’re really a studio building an agent. Quinn is coming to all our apps, and merchants can already talk to it from Claude, ChatGPT and Sidekick.

What’s next

The way I think about it is in three stages. Before, we answered every chat ourselves in Crisp. Today, Quinn answers the first questions inside the app and our team picks up whatever it can’t handle. Next, Quinn does the answering and we check its work.

We’re building a new inbox for that. A merchant asks something, Quinn looks at their setup, writes a reply, sends it and moves on to the next one.

The obvious worry is mistakes. A bot can’t really tell when it’s wrong, but a second agent reading over its shoulder can. In one week in September, Quinn passed 29 chats to a person. The same week, our internal agent went back through Quinn’s chats and flagged about 112 that it thought someone should look at. Not all of those were real problems, but it was enough to show us how much a bot misses about itself.

So in the new inbox, a second agent checks each answer as Quinn sends it. If something looks off, like Quinn saying a setting is on when the store says it’s off, it flags it for someone on the team and explains why. When a person does step in, Quinn writes a draft reply for them, and they can send it as is, edit it, or write their own.

The buddy, for everyone

Going back to the video game: for years only the merchants who reached out got a buddy, and usually only once something had already gone wrong. Now every merchant can have one from the start. It can help them set up, change settings, answer questions and look things up, at any time of day and in whatever language they write in.

What we’re betting on

I ended the talk with four things we believe.

1. People will keep using agents more, and faster than we expect.
2. Apps will stop being a fixed set of screens. Merchants will say what they want and the app will do it.
3. Merchants will ask more questions than ever, because asking is free and quick now. Most will be about setting things up properly.
4. Merchants will forgive AI for a mistake more easily than a person, as long as they got an answer quickly.

If you disagree with any of these, I’d really like to hear why. You can find me on LinkedIn or X.