Case study · AI agent · Calendar & email
Caldra AI: a scheduling agent
that reads the inbox.
Most scheduling tools live inside the calendar. The problem lives in the inbox. This is how I built an agent that works across both, what the architecture looks like, and the four things that broke in production before I caught them.
The problem
I tested nine AI scheduling assistants across a quarter of real meetings, and they split cleanly into two groups. One group is defensive: it guards your calendar, protects deep work, and reshuffles tasks. The other group is a booking page with better styling. Both operate entirely inside the calendar.
But the part that actually costs time happens before anything reaches the calendar. Someone emails proposing three times. You open the calendar, cross-check each one, discover two collide, reply with a counter-proposal, then create a tentative hold so you do not double-book while waiting. That loop is four context switches, and no tool I tested closed it.
Caldra was built for exactly that loop. The bet was that an agent with read access to both the thread and the calendar could collapse those four steps into one approval click.
The architecture
Four layers, and the model only appears in the third. Teams that start at the model layer build a demo that cannot be made reliable; the order below is the one that survived.
1. A normalized calendar model
Google Calendar and Microsoft Graph never reach business logic directly. An adapter per provider maps both into one internal event type: UTC instants with the originating timezone alongside, recurrences expanded into concrete instances inside a bounded window, and a single definition of what a conflict is. This is the least interesting layer and the highest-return one.
2. Inbox intent extraction
A cheap, fast classifier runs over every inbound thread and answers one question: is there scheduling intent here at all? Only the threads that pass reach a larger model, which extracts structured fields under a strict schema - proposed times, participants, duration, location, and how firm each proposal is. Nothing is parsed out of prose with regular expressions.
3. A constrained tool loop
The agent gets four tools: find free slots, read preferences, check conflicts, and propose. There is deliberately no tool that creates a calendar event. Candidate slots are scored against stated preferences in ordinary code before the model sees them, so the model ranks a shortlist rather than searching an availability grid.
4. The approval surface
A proposal, a one-line reason, and a single accept or reject. The approval performs the calendar write. Every proposal and every decision is logged, which doubles as the evaluation set for measuring whether the agent is getting better.
The stack
- Application
- Next.js, TypeScript, React
- Data
- Postgres, with a normalized internal event model
- Calendar
- Google Calendar API, Microsoft Graph
- Gmail API and Microsoft Graph mail, read-scoped
- Models
- A small classifier tier plus a larger extraction and reasoning tier
- Infrastructure
- Vercel, with a queue for backfill and webhook processing
Four things that broke first
Daylight saving moved recurring meetings
Recurrences created before a clock change and expanded after it shifted by an hour. The cause was caching expanded instances across the transition. The fix was storing UTC plus originating timezone and re-expanding rather than caching.
Revoked OAuth tokens looked like empty calendars
When a user changed their password, the refresh token died and the calendar query returned nothing. The agent read "no events" as "completely free" and confidently proposed a time on top of an existing meeting. Authentication failure is now a hard stop that surfaces to the user, never an empty result.
Signup backfill hit rate limits
Pulling calendar history on signup is exactly when throttling happens, and exactly the worst moment for the product to look broken. Backfill moved into a queue with exponential backoff, and the product became useful on partial data.
The agent kept proposing 8am on Mondays
Asked to find any valid slot, it did exactly that. This was not a prompt problem. The fix was scoring candidates against stated preferences in code first and handing the model a ranked shortlist instead of raw availability.
What I would keep, and what I would change
Keep: the no-write-tool rule. Removing the agent's ability to book anything eliminated the only failure mode that would have genuinely damaged trust, and the approval click cost users almost nothing. The normalized event layer, likewise, paid for itself many times over once a second provider arrived.
Change: I built the model layer before the calendar layer, and spent weeks debugging what looked like reasoning failures but were really Microsoft Graph recurrence semantics reaching the prompt. The deterministic foundation should come first every time.
Also change: I invoked the model on every calendar webhook early on, which was both slow and needlessly expensive. Invoking only on genuine ambiguity cut the call volume dramatically with no loss in usefulness.
Frequently asked questions
What is Caldra AI?
An AI scheduling assistant that works across the calendar and the inbox together. It reads email threads containing scheduling intent, cross-references them against connected Google and Microsoft calendars, and proposes times that respect the user's stated preferences. A person approves before anything is written to a calendar. I built it, and I use it daily for my own scheduling.
What was the hardest engineering problem?
Not the AI. It was reconciling Google Calendar and Microsoft Graph into a single event model. The two providers disagree on recurring event expansion, on how free/busy is exposed, on all-day event timezone handling, and on what constitutes a conflict. Until every provider difference was absorbed by an adapter layer, those differences leaked upward into the agent's reasoning and produced bugs that looked like model failures but were really API semantics.
Why can the agent not book meetings directly?
Because that is the design, not a limitation. The agent has no tool that writes to a calendar. It can search availability, read preferences, check conflicts, and propose - and the proposal writes to Caldra's own store, not to Google or Microsoft. A human approval performs the actual write. This removes the entire class of catastrophic failure at the cost of roughly one second of user effort, which turned out to be a trade nobody complained about.
What models does it use?
A two-tier setup. A small, fast model runs the first-pass classifier over every inbound thread, answering only whether the thread contains scheduling intent at all. Threads that pass go to a larger model for structured extraction and for the proposal loop. Splitting the work this way is what makes the running cost reasonable, because the cheap model handles the overwhelming majority of the volume and the expensive one only sees candidates.
Can you build something similar for us?
Yes, and the honest first question is whether you should buy instead. If your scheduling logic is your product, has to live inside an application you already own, or genuinely does not fit any SaaS tool, a custom build makes sense. Otherwise a twenty-dollar-per-month tool wins on economics. I will tell you which case you are in on the call rather than after the invoice.