How do you test an AI agent that uses a calendar?

Write the expected event before asking the agent

A calendar assistant can produce a convincing confirmation while choosing the wrong calendar or misunderstanding the time. Start the evaluation with a small written specification that a second person could check without reading the conversation. This guide proposes a test plan using Google Calendar documentation as a concrete reference. We have not executed these scenarios as a BotBento integration test, and BotBento remains in development.

Our fictional task is to create one event called TEST: sketch review, on 14 September 2026, from 10:00 to 10:30 in Asia/Amman, on a private calendar called Agent Sandbox. It has no attendees, recurrence, attachments or conferencing. The calendar name is a human label; record its actual identifier when setting up your own fixture.

Choose a disposable test account and calendar you control. Keep real appointments and other people's addresses out of the initial test. Record the initial state so you can distinguish a new event from an old fixture left by a previous run. Use a different run marker when starting a genuinely new test, and retain the previous marker when investigating an uncertain result.

Check the calendar and time in the saved result

Google's creation guide distinguishes the destination calendar identifier from the event body. The special value primary refers to the signed-in user's primary calendar. It also distinguishes timed events, which use dateTime fields, from all-day events, which use date fields. These details make useful assertions for a calendar-agent test.

For our fixture, require the agent's proposed destination to resolve to Agent Sandbox. Inspect the exact date, start, end and time zone before the write. Afterward, use Google's Events: get operation with the returned event ID and intended calendar ID to retrieve the saved event. Compare the result with your original specification, and inspect the event in the calendar interface as a separate human readback.

Grade each assertion independently: correct destination, correct date, thirty-minute duration, correct zone, no guests and one event. A saved event with the wrong time is a failed task even if the API request succeeded. Conversely, if the event exists but the agent reports failure, record that mismatch too; it affects whether a person will repeat the request.

Sources: Create events, Events: get.

Test an unclear instruction and a denied write separately

Now change one input at a time. For an ambiguity test, omit the time zone and ensure the test conversation supplies no agreed default. Our proposed acceptance rule is that the agent asks for the missing zone before saving. That rule is an evaluation choice for this fixture, not a Google API requirement. A different application may have an explicit user-approved default; test that default visibly instead of relying on an assumption.

For the permission test, use a separate connection whose granted scopes you have verified are read-only. Google's Events: get accepts calendar.events.readonly, while Events: insert lists write-capable scopes and does not include that read-only scope. Google's consent guidance recommends requesting the minimum access required for the application.

Ask the read-only agent to create the same disposable event. The desired application behavior is to explain that it cannot save the event with its current access, retain any useful draft and leave the calendar unchanged. If the application blocks the write before contacting Google, record an application-level denial. Test the provider's denial separately in the integration harness. Neither result should be labelled a successful calendar write.

Sources: Events: get, Events: insert, Configure the OAuth consent screen and choose scopes.

Make an uncertain write distinguishable from a fresh request

In a controlled integration harness, model a write whose response is lost after the provider accepts it. Do not interrupt a real appointment to manufacture this condition. Google's creation guide describes supplying a compliant event ID to help prevent duplicate creation when a failure happens after Calendar has executed the request. Use the documented ID format; an arbitrary test label is not necessarily a valid event ID.

Our proposed check is that the integration preserves the action's identity, inspects the intended event and compares its fields before deciding what to retry. An existing ID with different contents is a discrepancy to investigate. Record the evidence that resolves the uncertainty instead of turning every timeout into a new insertion.

Keep the first fixture guest-free. Google's insert reference documents notification options and cautions about side effects of suppressing updates. Testing invitation delivery deserves its own controlled recipient and delivery evidence; an event appearing in the organizer's calendar does not establish that an invite reached someone else's inbox.

Sources: Create events, Events: insert.

Revoke the test connection and inspect the next attempt

Run revocation checks only with an isolated test user and OAuth project. Google's OAuth documentation says revocation removes the user's granted scopes for the project and invalidates its issued access and refresh tokens across that project's clients. It also notes that revocation can take time to have full effect. Revoking a production connection merely to test a calendar feature can therefore affect more than that feature.

After revoking the test grant through the supported account or application flow, record when it happened and inspect a subsequent authenticated request. Do not declare the test passed from the revocation response alone. Distinguish an actual provider read from content the application already cached; displaying an old event is not evidence of continuing access.

The behavior we want to evaluate is understandable recovery: the agent identifies the unavailable connection, does not claim a new write succeeded and requires reconnection before resuming dependent work. If access still works during propagation, keep the check unresolved and observe it again within a bounded test period. Reconnect deliberately for any remaining cleanup; do not silently restore the revoked grant.

Sources: Using OAuth 2.0 for Web Server Applications.

Keep a small acceptance record and clean up the fixture

For each scenario, save the intended event, the connection's tested access level, the observed calendar result and the agent's final statement. Keep tokens and unrelated event contents out of the report. Name any untested behavior, such as recurring events, daylight-saving transitions, attendee responses or another provider. Passing this narrow plan does not prove those separate cases.

Finally, remove only the disposable records belonging to the test, using their retained identifiers and an authorized cleanup connection. Verify cleanup and preserve the short result record. The useful outcome is a set of specific, reproducible observations that lets you decide whether this calendar task is ready for its intended use.

Keep reading

• MCP tool permissions: what to check before connecting a bot
• What should an AI agent record after each run?