How to Evaluate an AI Strategy Firm (Including Us)

You got the mandate. Now you need help. The problem is you’ve probably never hired for this before.

The RFP templates from your last ERP implementation don’t fit. The procurement team wants three bids and a scoring matrix. The big consulting firms are sending decks with 40 slides and a team of 12. The boutique shops are showing you demos of things they built for someone else. Everyone sounds smart. Everyone has case studies. Everyone says they’re different.

Here’s what we think you should actually ask. We’re publishing this because the best way to earn your trust is to help you evaluate everyone honestly. Including us.

Seven questions. Ask all of them.

1. Who does the work?

Not who sells it. Not who presents it. Who actually sits in the room with your team, maps your systems, and writes the deliverables?

At large firms, the partner who sold the engagement disappears after kickoff. The work gets done by analysts two years out of school. They’re smart. They’re also learning on your budget.

What a good answer sounds like: A name. A background. A LinkedIn profile you can check. The person who does the assessment should be the person who presents the findings.

What a red flag sounds like: “We’ll assign a team based on availability.”

2. What do you deliver, specifically?

Not capabilities. Not methodologies. Deliverables. Documents your team can hold, read, and execute from.

Every firm will tell you they do “AI strategy.” Ask them what you walk away with after 30 days. If the answer is a slide deck and a statement of work for phase two, that’s not a strategy. That’s a sales funnel disguised as consulting.

What a good answer sounds like: A numbered list. Assessment, decision matrix, architecture blueprint, roadmap. Defined scope, defined timeline, defined format.

What a red flag sounds like: “It depends on what we find.” That’s not flexibility. That’s not having a methodology.

3. Have you built what you’re advising on?

There’s a difference between someone who studies AI and someone who builds with it. Advisors who read papers and attend conferences will give you a strategy that sounds right. Builders who’ve shipped systems and watched them fail will give you one that works.

Ask them: what did you build last month? Not last year. Last month. AI moves fast enough that experience from 2024 is already dated.

What a good answer sounds like: Specific systems, specific outcomes, recent work. Failures they learned from, not just successes they’re proud of.

What a red flag sounds like: Frameworks and matrices without implementation stories.

4. What’s your relationship with vendors?

Every AI strategy will eventually recommend specific platforms, tools, or services. You need to know whether the recommendation is independent.

Some firms have partnership agreements, referral fees, or reseller relationships with cloud providers and AI platforms. That doesn’t make their advice bad. But you deserve to know about it before you take it.

What a good answer sounds like: “We have no vendor partnerships. Our recommendations are based entirely on your requirements.” Or, if they do have partnerships: “We partner with X. Here’s why, and here’s how we manage the conflict.”

What a red flag sounds like: Avoiding the question. Changing the subject to “best-of-breed” without naming names.

5. What happens if AI isn’t the answer?

This is the question that separates strategists from salespeople. A firm that only sells AI solutions will always find an AI problem. A firm that solves business problems will sometimes tell you that a better-configured CRM, a cleaned-up data pipeline, or a simpler automation is the right move.

Your AI strategy firm should be willing to lose the engagement by being honest.

What a good answer sounds like: “We’ve told clients not to pursue AI. Here’s when and why.”

What a red flag sounds like: “AI can help with everything.” No, it can’t.

6. Can my team execute your plan without you?

The deliverable should be yours. Not a proprietary framework that requires their ongoing involvement to interpret. Not a platform that needs their consultants to maintain. Not a roadmap written in jargon that only their team understands.

You’re buying a map, not a guide service. The map should work even if you never speak to them again.

What a good answer sounds like: “The roadmap is designed for your team to execute independently. We’re available if you want support, but you don’t need us.”

What a red flag sounds like: “We recommend a retainer for the implementation phase.” Before you’ve even seen the strategy.

7. What does it cost, and how is it scoped?

Open-ended engagements with hourly billing create an incentive to find more problems. Fixed-scope engagements with defined deliverables create an incentive to deliver.

Ask for the total cost before signing. Ask what’s included and what’s not. Ask what happens if the scope changes mid-engagement.

What a good answer sounds like: A number. A timeline. A list of deliverables. A change order process for scope adjustments.

What a red flag sounds like: “We’ll scope it after discovery.” That means discovery is the product and you’re already paying for it.

How we answer these questions

We publish this because we think it applies to us too.

1. Who does the work? Bryant Herrman. Every engagement. Nearly 20 years of enterprise technology at Kaiser Permanente, PricewaterhouseCoopers, and as a founder. You work with the principal.

2. What do we deliver? Four documents in 30 days: Discovery Assessment, Platform Decision Matrix, Architecture Blueprint, and Strategic Roadmap. Defined before we start.

3. Have we built it? We build with AI daily. Our practice runs on AI workflows. We don’t advise from the outside. We operate from the inside.

4. Vendor relationships? None. Zero partnerships, referral fees, or reseller agreements. Our recommendations are independent.

5. What if AI isn’t the answer? We’ll tell you. We’ve done it before. If the fix is cleaning up your data or reconfiguring your existing tools, that’s what we recommend.

6. Can your team execute without us? That’s the goal. The roadmap is yours. It’s written so your team can run it. We’re available for support, but independence is the design.

7. What does it cost? Fixed scope, quoted before we start. No hourly billing. No open-ended engagements. No surprise invoices.

What to do next

Use these questions on every firm you talk to. Write down the answers. Compare them side by side. The firm that gives you the clearest, most specific answers is probably the one that’s done this before.

If you want to hear our answers in person, book 30 minutes. We’ll tell you what we see and whether we can help. If we can’t, we’ll tell you that too.

Now you know how to evaluate. The next question is whether your organization is actually ready. That’s a different problem, and we wrote about it: What “AI-Ready” Actually Means.

Bryant Herrman is the founder of Merivant, an AI-native strategy firm based in Los Angeles.