A deep dive into the playbook, lessons learned, and brutal truths about deploying 20+ AI agents into production — that actually work.
During Dreamforce Jason (CEO) and Amelia (Chief AI Officer) at SaaStr dropped by Qualified’s office for a deep dive on its AI Agents and top learnings with Kraig Swensrud (Founder & CEO, Qualified)
The Bottom Line Up Front
If you’re a CMO, CRO, or founder and you haven’t deployed at least one AI agent by Halloween 2025, you’re already behind. Not “might fall behind” — you’re already behind. “Every VC I know where a startup hasn’t made the jump yet has given up hope on that company. That’s not hyperbole. That’s the market reality.” per Jason.
But here’s what under-discussed training is more important than picking the perfect vendor. We started 2025 with zero AI agents at SaaStr. Now we have 20 in production. The secret wasn’t finding magical tools — it was investing 30 days of deep training upfront, then maintaining an hour every single day.
Here’s exactly how we did it, what worked, what failed, and the framework you need to deploy agents that actually drive revenue.
The New Budget Reality: Why Traditional SaaS Playbooks Are Dead
Let’s start with the uncomfortable truth about 2025 budgets:
Traditional SaaS budgets are frozen. CEOs are going around the table telling every functional head to cut 20-30% of their SaaS apps. Half of incremental budgets are going to price increases — Salesforce raised prices 8% this year, others 6-7%. When your IT budget is growing 6% but your core vendors are raising prices 7-8%, where’s the room for another business process workflow app?
But AI budgets are exploding. Business software is growing faster than it has ever before — if you tap into AI budget. It’s the only incremental budget most companies have. Nobody is putting more money into old SaaS software. It’s all going into AI.
This creates a tale of two cities:
Classic SaaS is geriatric ... but ...
B2B software powered by AI is exploding
If you’re selling the way you sold in 2021, with the 2019 Marketo playbook, there’s no budget for you. The playbook doesn’t work. But tell that to anyone on fire with AI — everything works. Outbound works. Events work. Meeting with customers works. If anyone wants to buy your product, it all works.
The Vendor Selection Myth: Training Trumps Everything
Here’s the question Jason gets constantly on LinkedIn: “What’s better — Replit or Lovable? Which AI SDR platform is best? What AI tool is best for RevOps?”
Wrong question.
Here’s what Jason learned after deploying 20 agents: Pick a leader and go deep. Training is more important than vendor selection.
There are very few agentic products with any level of complexity where you can just flip a switch. Take Qualified — it set up seven appointments for us on its own. Sounds great, right? Put it in a case study. But it wouldn’t have happened if we put zero minutes of training into it.
The New Deployment Reality
Remember how people bought enterprise software before AI? You’d buy it from a sales rep. You might hire Accenture or Blue Wolf to deploy it over a year. You’d hope and pray it worked like the sales rep told you.
No one will tolerate that in AI this year. Time to value often has to happen before you even get a contract signed.
Jason is 100 days into Replit but probably 200 hours expert now to make it great. That’s a sea change in how we use software. Companies are used to buying from a people person and scaling up over time with humans doing all the work. Those days are behind us.
The Training Investment Framework
For any agent you deploy:
Commit 30 days upfront — train it every single day
Then commit weekly — every week after that
Budget 1 hour daily ongoing — this is your new normal
Most people don’t know how to do this. But if you don’t train it, you won’t get ROI. Period.
The SaaStr Agent Journey: From 0 to 20 in 10 Months
Agent #1: Jason AI (The Horizontal Foundation) aka “Digital Jason”
SaaStr started with Delphi, building a clone of Jason — “Jason AI.“ It ingested all of SaaStr content (20 million words, 1,000 YouTube videos, all tweets, all LinkedIn posts). It actually broke the ingestion engine at first, but once it worked, it was magical.
Key insight: SaaStr started with a horizontal agent that could do a little bit of everything. No complex workflows. Just digital Jason answering founder questions. Training was broad, but not especially deep. Perfect for a general agent, answering general questions.
What happened: People started using it for things we didn’t envision. They used it for support questions about attending our events, sponsor questions, booth logistics. Classic issue with AI agents — if it’s good, people will start using it for everything. If they trust it, usage expands.
Then it started to hallucinate on event-specific questions. So the team uploaded the prospectus. With that additional context and training, it got good. Maybe 20% as good as Qualified, but 20% is infinitely better than zero.
Before: Pre-Fin Intercom with week-long response times
After: Instant answers, 90% accuracy on customer questions
The Stair-Step Strategy That Actually Worked
Here’s the deployment sequence that worked for SaaStr:
Step 1: Horizontal Agent (Delphi/Jason AI)
Easiest to deploy
Least training required
Built confidence
Got an early win
Step 2: Outbound SDR (Artisan)
Lower risk than inbound
SaaStr’s SDRs wouldn’t do outbound anyway
Better to send a better email via agent than mediocre human
Less risk than trusting agents with precious inbound leads
Step 3: Inbound Qualification (Qualified)
Higher complexity
More critical to get right
But SaaStr’s qualification process was nearly broken
Low bar made deployment easier
Step 4-20: Specialized Agents
Email campaigns
Website traffic deanonymization
Follow-up sequences
Support ticketing
AI speaker content review
AI VC
Why Stair-Stepping Matters
If you fail with your first agentic thing, you won’t know what went wrong. You’ll get frustrated. You won’t build the confidence to keep going.
SaaStr accidentally stair-stepped their deployment, and it’s why they succeeded. They gained confidence with each win. By the time they deployed Qualified for inbound (higher stakes), the team knew how to train agents properly.
The framework:
Use horizontal agents where bar is low, where there is nothing already and you have some strong content to train it on. But it doesn’t need to be a vast amount of content. It just has to be useful and relevant.
Use specialized agents at first for areas with zero human coverage. Because the bar is low for success.
Save the hardest, most critical deployments for when you have 2-3 agents under your belt.
The “Layup Roles”: Where to Start Right Now
The biggest question: where do I actually start?
The layup roles are where there’s a trustworthy tool that exists, but work is just not getting done in your org.
There are lots of reasons work doesn’t get done:
Tough to hire in that area
Tough geography for hiring
Not your DNA (product-centric companies suck at sales, sales-centric companies suck at product)
High turnover
Junior team members who don’t know your product/market/competitors
Don’t try to replace stuff that’s working. Look at where there’s literally no one doing the work.
The Proven Starting Points
#1: Customer Support (The Easiest Layup)
Go try your own support as a founder or executive right now. Nine times out of ten, you’ll cry at how bad it is. Submit a P1 issue. See if you get “We’ll get back to you in a week” or “I don’t understand the problem.”
That’s where to start. The bar is so low, your agent doesn’t have to be amazing. It just has to be better than terrible.
We saw this at SaaStr: Went from Intercom (week response times) to instant accurate answers in 90% of cases. The other 10%? The agent knew to escalate.
#2: Outbound SDR/BDR
For us, this was lower hanging fruit than inbound. Why? Our SDRs couldn’t (or wouldn’t) do outbound consistently. The bar: send better emails than mediocre humans were sending — or in our case, not sending.
Less risky than inbound because you’re not trusting agents with precious inbound leads. You’re just trying to generate pipeline that wasn’t being generated before.
#3: Inbound Qualification (Higher Complexity, But Massive ROI)
SaaStr’s process before Qualified: come to website, fill out contact form, hope over the next week a human gets back to you. Maybe they follow up, maybe they don’t. Round-robin meant inconsistent quality.
The Qualified deployment results:
7 appointments set autonomously in the first weeks
~100 tickets sold to London event in 6 weeks
Overnight coverage (Pacific team sleeping while UK prospects active)
Better qualification than our human process
The bar wasn’t “be better than a great team of 10 product experts with 5-minute SLAs.” The bar was “be better than broken.” That made deployment possible.
What About the “Hero Purchase”?
Here’s the failure mode I see CMOs make constantly: they want a hero purchase. They want to go to the CEO and say “I bought AI.”
This will probably fail. Why?
That hero purchase is likely aligned with something you’re already decent at
The bar is high because it’s already working
It’s your first agent, so you don’t know how to train it
You’re trying to make something 20% better instead of fixing something that’s 0% done
Better approach: Leave that hero app on the shelf. Deploy something where work literally isn’t happening. Get a win. Build confidence. Then deploy the hero app when you know what you’re doing.
If AI is only going to make something 20% better, maybe 2025 isn’t the year. Maybe that’s a 2026 project.
The Daily Reality: What Managing 20 Agents Actually Looks Like
The hour Amelia spends with agents every morning used to be spent with people.
Before: One-on-ones with SDRs and BDRs, going through accounts, providing context on leads, giving them history on prospects.
Now: Amelia wakes up, checks each agent:
Qualified — see meetings booked overnight (London prospects while we sleep), review conversations, give context to sales team
Artisan — review outbound performance, adjust sequences
Delphi — check Digital Jason AI conversations, refine responses
Agent Force — review support tickets, catch edge cases
Cycle through remaining agents
Total time: 1 hour
What’s different:
Higher cognitive load — you have to think more, use more brain cells
Compounds over time — unlike humans who leave, every hour of training pays forward
Agents don’t cry — they don’t get emotional, but they require deep thinking
You can do more — now expanding to email campaigns, deanonymization, follow-up sequences
The Early Days Reality
First 30 days: Way more than 1 hour. You’re setting everything up, fixing data issues (our Salesforce data was terrible — agents expose that), connecting systems, training workflows.
After 30 days: Maintenance mode at ~1 hour daily, but you’re constantly expanding use cases.
The Compound Effect
With humans: Train for months, they get good, they leave, repeat.
With agents: Every hour of training compounds. They get better every day. They don’t forget. They don’t quit. They work 24/7.
Example: Qualified handles UK prospects overnight while Pacific team sleeps. Those conversations used to just... not happen. Now they do, and they’re high quality.
The Brutal Truths About Team Reactions
For Founders: Some People Will Leave When You Roll Out AI Agents. That’s Okay
Amelia was talking to a founder this week at Dreamforce. She has some agents deployed but is worried about rolling out to her 50-person sales team. “I think they might revolt. Some might quit.”
Amelia’s answer: That’s okay. Let them.
It’s almost 2026. If your team can’t get over the hurdle, some may quietly leave. You cannot forsake the future of your company to avoid hurting feelings.
Rip the band-aid off. Do it the right way — show them they can be empowered with AI, crush quota with AI, do more with AI. The best ones will see this and get excited. They should be roaring to get AI going.
If you see people hiding in the corner, maybe it’s okay if they quietly weed themselves out.
The Exposure Problem
SaaStr had someone leave their team the day they hooked up Momentum (their conversation recorder and GTM tool). Why? It Slacks summaries after every sales call and notifies Amelia when opportunities open or meetings book.
This rep had zero activity. The system exposed it instantly. They quit that day. You can’t hide anymore.
For Individual Contributors: Deploy It Yourself or Find a New Job
Jason recently had a CRO he’s worked with for years reach out in a panicked email: “I need to learn AI. I’ll do anything. I’ll intern for you. I’ll hang out in Amelia’s office.”
Jason’s advice: Don’t learn it. DO it.
Buying a subscription to Claude does not count as learning AI. Here’s what counts:
Pick one agentic product — simplest possible use case
Buy it yourself — don’t delegate to your team
Set it up yourself — okay to get help with CRM connectors, but own the deployment
Train it yourself — take your time, do something simple
QA and test yourself — hands on keyboard
If you deploy an agent yourself, train it, QA it, test it — you will be ahead of 90% of the world.
The CMO Who Jason Couldn’t Help
Another CMO Jason loves reached out after many years. Jason had gotten her the last two jobs. She was ready for number three.
Jason’s response: “I got nothing for you.”
Why? She doesn’t know this stuff yet. She hasn’t deployed an agent. She can’t tell Jason how training went, what the issues were, what worked.
Here’s the weird thing: There is much more demand for great CMOs than CMOs think. Jason knows 100 CEOs today who are desperate for a great marketer. They’ll do anything.
But they do not want someone running the 2019 Marketo playbook. Zero interest in hiring that person.
Jason’s advice to her: “Go deploy an agent. Tell me how it worked. Tell me how training went. Tell me the issues. Come back to me. I’ll get you two jobs.”
The ROI Reality: What Results Actually Look Like
Qualified Results (6-8 Weeks of Deployment)
7 autonomous meeting bookings per week in early weeks
~100 tickets sold to London event so far out of ~2,000 total (5% conversion via agent)
24/7 coverage — overnight conversations with UK prospects while Pacific team sleeps
Better qualification than human process — knows when to escalate, provides context to sales team
Artisan Results (Outbound)
Better email performance than human SDRs who weren’t doing outbound
Consistent execution — no more “round robin to rep who doesn’t follow up”
Scaling sequences we could never scale with humans
Overall Impact
From 0 to 20 agents in 10 months
~12 core agents in production running go-to-market
1 hour daily maintenance (down from way more in early days)
Work that literally wasn’t getting done now happening 24/7
The brutal truth: Half the time, SaaStr’s sales team wasn’t following up with inbound leads. At least half the time. With round-robin, good reps followed up instantly. Mediocre reps... maybe, maybe not.
Now: Instant response, 24/7, consistent quality, full context provided to sales team.
The Deadline Reality: How Fast Do You Need to Move?
For Startups: Now
If you’re a startup and you don’t have anything disruptive in production by now, you’re already behind.
Every VC Jason knows, every investor he knows — if the startup hasn’t made the jump yet, they’ve lost hope. Not “losing hope” — they’ve LOST hope.
Why?
You could make excuses when ChatGPT came out in 2023. There were so many hallucinations. We could all make fun of it.
2024? You could be a critic. Maybe you’d start losing deals to hot AI companies, but at least you could have a principled position that this stuff wasn’t going to work.
2025? There’s no excuse.
Everything’s great now. Even mediocre software got 5x better when you put Claude 4 in. Replit took 10 years to get to $1M ARR. They added Claude 4, now they’re almost $200M in one year. Same IDE. Same software.
If you’re not in market with a disruptive AI agent by the end of the quarter:
You need a brand new team. Maybe 80% have to go. Give them a nice Thanksgiving bonus and a turkey and tell them they’ve got to go. Because you’ve had 10+ months.
For Enterprises: It Depends (But Don’t Wait Long)
If your business is fine, if growth is as good or better than 12 months ago, if you’re in a traditional industry where everything’s working — you can be a late adopter.
There are late adopter advantages in AI:
Tools get better fast (Replit is epically better than 100 days ago)
Training resources improve
Best practices emerge
Edge cases get solved
But if you’re facing external pressure, if competition is heating up, if you’re losing deals — you need to move now.
The New Software Buying Reality
In the old SaaS days, a customer would come in and say “I need this feature.” You’d promise it in the contract even though it didn’t exist. You’d brute force it over 12 months.
Those days are dying in AI.
Don’t promise me it’ll be here in a year. Show me the agent works today. If it’s not ready, I don’t blame you. I get it’s early. Come back to me when it works. It might be ready in three months.
Time to value has to be before contract signature in many cases now.
The Framework: Your 90-Day Deployment Plan
Days 1-30: Foundation and First Agent
Week 1: Pick Your Layup
Audit what’s NOT getting done (support? outbound? qualification?)
Pick the simplest use case with lowest bar
Select a leading vendor (doesn’t have to be THE leader, just A leader)
Week 2-3: Setup and Integration
Buy the tool yourself (don’t delegate if you’re the exec)
Set up core integrations (CRM, email, support systems)
Clean your data (this will take longer than you think)
Build basic workflows
Week 4: Daily Training
Train the agent every single day
Test edge cases
Refine responses
Build your QA process
Success metric: One working agent handling real work by day 30
Days 31-60: Expansion and Specialization
Week 5-6: Monitor and Optimize
Spend 1 hour daily with your agent
Review conversations/outputs
Provide additional context
Expand use cases slightly
Week 7-8: Agent #2
Pick second use case (slightly harder)
Apply lessons from Agent #1
Faster setup because you know what you’re doing
Stair-step to more complexity
Success metric: Two agents in production, 1 hour daily maintenance
Days 61-90: Scale and Specialize
Week 9-10: Vertical Agents
Move from horizontal to specialized
Deploy agents for specific workflows
Start tackling harder use cases
Build your agent management system
Week 11-12: Team Enablement
Train your team on working with agents
Build handoff processes
Create monitoring systems
Document what works
Success metric: 3-5 agents in production, team enabled, clear ROI
The Future: What’s Coming Next
Voice and Video Agents (Live at SaaStr Now)
SaaStr is deploying Amelia AI with voice and video. Not just text chat — full video agent of Amelia on their event sites.
Why? Trust has been built through chat. Now the team can do more:
Overnight support with full video presence
Event co-pilot for attendees
Session recommendations
Networking coordination
“Real Amelia is asleep, but Amelia AI is awake”
The funky part: People walk up to real Amelia now and say “You’re that Amelia AI!” The line between digital and real is blurring.
The Compounding Use Cases
Once you have 5-10 agents working well, new use cases emerge constantly:
Email deanonymization of website traffic
Follow-up campaigns for unconverted prospects
Pre-meeting research and briefings
Post-meeting follow-up and deal progression
Internal knowledge management
Onboarding automation
The cognitive load increases, but so does output. It’s exhausting in a different way than managing people, but it’s more productive.
The Market Reality: Everyone’s In Market
Traditionally, you’d hope a prospect would be in market every 5-6 years for a big buy. Why? Takes a year to get going, second year to scale, third year to get frustrated, fourth/fifth year to start looking at new solutions.
Right now, everything’s in market again.
This hasn’t happened since maybe when the web started. Everyone either has AI budget or is under pressure from their board/CEO to show AI innovation. It’s the only incremental budget.
For vendors: This is the opportunity of a lifetime
For buyers: The tools will never be better-priced or more hungry for your business
The Final Word: Just Rip the Band-Aid Off
Let’s be brutally direct:
If you’re a go-to-market executive and you haven’t deployed an AI agent yourself by now, you’re unemployable.
Not “will be unemployable” — you ARE unemployable. Jason has CMO friends he loves, who he’s gotten jobs for multiple times, who he now has nothing for. Why? They don’t know this stuff yet.
The Simple Advice:
Don’t learn AI — DO AI
Pick one agent, simplest possible use case
Deploy it yourself (hands on keyboard)
Train it for 30 days
Spend 1 hour daily after that
You will be ahead of 90% of the world.
The agents don’t cry. They work 24/7. Every hour of training compounds. They get better every day, they don’t forget, they don’t quit.
But only if you actually do it.
It’s almost 2026. Rip the band-aid off. Let’s go.
Key Takeaways
Training > Vendor Selection — Pick a leader and go deep, training matters more than choosing the perfect tool
Stair-Step Your Deployment — Start simple, build confidence, then tackle harder use cases
Find the Layup Roles — Deploy where work isn’t getting done, not where it’s already working
Budget 30 Days + 1 Hour Daily — This is your new reality, embrace it
Halloween Deadline for Startups — If you’re not in production now, you need a new team
Do It, Don’t Learn It — Hands on keyboard deployment is the only way to actually learn
Some People Will Leave — That’s okay, you can’t forsake your company’s future
The Compound Effect is Real — Unlike humans, agents get better every day forever
The future belongs to companies that move now. Not next quarter. Now.
Want to see Jason AI in action? Go to SaaStr.com and click the bottom right corner. Want to see how SaaStr deployed 20 agents? Follow Jason Lemkin on LinkedIn or come with your questions LIVE to SaaStr AI London on Dec 1-2!!
