feat: cal.ai v1.2.0 (#11868)

* bump nextjs version in ai

* lowercase username and email

* onboarding email

* direct user to install app if not installed

* multiple suggested times for link flow

* summary of context prompt engineering

* specify the @username nuance and discourage Ids

* v1.2.0

* Update README

* Change title

* simplify and improve booking link flow

* add build:ai to package.json

* better onboarding copy

* onboarding touches

* remove console logs and temp hacks

* remove env vars in app store and token in AI app

* invited user id should be string

---------

Co-authored-by: tedspare <ted.spare@gmail.com>
This commit is contained in:
DexterStorey 2023-10-14 04:52:24 -04:00 committed by GitHub
parent a5fa2ef8d0
commit 3047b5319b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 231 additions and 106 deletions

View File

@ -6,6 +6,9 @@ FRONTEND_URL=http://localhost:3000
APP_ID=cal-ai
APP_URL=http://localhost:3000/apps/cal-ai
# This is for the onboard route. Which domain should we send emails from?
SENDER_DOMAIN=cal.ai
# Used to verify requests from sendgrid. You can generate a new one with: `openssl rand -hex 32`
PARSE_KEY=

View File

@ -1,12 +1,12 @@
# Cal.com Email Assistant
# Cal.ai
Welcome to the first stage of Cal.ai!
Welcome to [Cal.ai](https://cal.ai)!
This app lets you chat with your calendar via email:
- Turn informal emails into bookings eg. forward "wanna meet tmrw at 2pm?"
- List and rearrange your bookings eg. "Cancel my next meeting"
- Answer basic questions about your busiest times eg. "How does my Tuesday look?"
- List and rearrange your bookings eg. "clear my afternoon"
- Answer basic questions about your busiest times eg. "how does my Tuesday look?"
The core logic is contained in [agent/route.ts](/apps/ai/src/app/api/agent/route.ts). Here, a [LangChain Agent Executor](https://docs.langchain.com/docs/components/agents/agent-executor) is tasked with following your instructions. Given your last-known timezone, working hours, and busy times, it attempts to CRUD your bookings.
@ -14,7 +14,7 @@ _The AI agent can only choose from a set of tools, without ever seeing your API
Emails are cleaned and routed in [receive/route.ts](/apps/ai/src/app/api/receive/route.ts) using [MailParser](https://nodemailer.com/extras/mailparser/).
Incoming emails are routed by email address. Addresses are verified by [DKIM record](https://support.google.com/a/answer/174124?hl=en), making it hard to spoof them.
Incoming emails are routed by email address. Addresses are verified by [DKIM record](https://support.google.com/a/answer/174124?hl=en), making them hard to spoof.
## Getting Started
@ -22,27 +22,39 @@ Incoming emails are routed by email address. Addresses are verified by [DKIM rec
If you haven't yet, please run the [root setup](/README.md) steps.
Before running the app, please see [env.mjs](./src/env.mjs) for all required environment variables. You'll need:
Before running the app, please see [env.mjs](./src/env.mjs) for all required environment variables. Run `cp .env.example .env` in this folder to get started. You'll need:
- An [OpenAI API key](https://platform.openai.com/account/api-keys) with access to GPT-4
- A [SendGrid API key](https://app.sendgrid.com/settings/api_keys)
- A default sender email (for example, `ai@cal.dev`)
- The Cal.ai's app ID and URL (see [add.ts](/packages/app-store/cal-ai/api/index.ts))
- A default sender email (for example, `me@dev.example.com`)
- The Cal.ai app's ID and URL (see [add.ts](/packages/app-store/cal-ai/api/index.ts))
- A unique value for `PARSE_KEY` with `openssl rand -hex 32`
To stand up the API and AI apps simultaneously, simply run `yarn dev:ai`.
### Agent Architecture
The scheduling agent in [agent/route.ts](/apps/ai/src/app/api/agent/route.ts) calls an LLM (in this case, GPT-4) in a loop to accomplish a multi-step task. We use an [OpenAI Functions agent](https://js.langchain.com/docs/modules/agents/agent_types/openai_functions_agent), which is fine-tuned to output text suited for passing to tools.
Tools (eg. [`createBooking`](/apps/ai/src/tools/createBooking.ts)) are simply JavaScript methods wrapped by Zod schemas, telling the agent what format to output.
Here is the full architecture:
![Cal.ai architecture](/apps/ai/src/public/architecture.png)
### Email Router
To expose the AI app, run `ngrok http 3000` (or the AI app's port number) in a new terminal. You may need to install [nGrok](https://ngrok.com/).
To forward incoming emails to the Node.js server, one option is to use [SendGrid's Inbound Parse Webhook](https://docs.sendgrid.com/for-developers/parsing-email/setting-up-the-inbound-parse-webhook).
To forward incoming emails to the serverless function at `/agent`, we use [SendGrid's Inbound Parse](https://docs.sendgrid.com/for-developers/parsing-email/setting-up-the-inbound-parse-webhook).
1. [Sign up for an account](https://signup.sendgrid.com/)
2. Go to Settings > [Inbound Parse](https://app.sendgrid.com/settings/parse) > Add Host & URL.
3. For subdomain, use `<sub>.<domain>.com` for now, where `sub` can be any subdomain but `domain.com` will need to be verified via MX records in your environment variables, eg. on [Vercel](https://vercel.com/guides/how-to-add-vercel-environment-variables).
4. Use the nGrok URL from above as the **Destination URL**.
5. Activate "POST the raw, full MIME message".
6. Send an email to `<anyone>@ai.example.com`. You should see a ping on the nGrok listener and Node.js server.
7. Adjust the logic in [receive/route.ts](/apps/ai/src/app/api/receive/route.ts), save to hot-reload, and send another email to test the behaviour.
1. Ensure you have a [SendGrid account](https://signup.sendgrid.com/)
2. Ensure you have an authenticated domain. Go to Settings > Sender Authentication > Authenticate. For DNS host, select `I'm not sure`. Click Next and add your domain, eg. `example.com`. Choose Manual Setup. You'll be given three CNAME records to add to your DNS settings, eg. in [Vercel Domains](https://vercel.com/dashboard/domains). After adding those records, click Verify. To troubleshoot, see the [full instructions](https://docs.sendgrid.com/ui/account-and-settings/how-to-set-up-domain-authentication).
3. Authorize your domain for email with MX records: one with name `[your domain].com` and value `mx.sendgrid.net.`, and another with name `bounces.[your domain].com` and value `feedback-smtp.us-east-1.amazonses.com`, both with priority `10` if prompted.
4. Go to Settings > [Inbound Parse](https://app.sendgrid.com/settings/parse) > Add Host & URL. Choose your authenticated domain.
5. In the Destination URL field, use the nGrok URL from above along with the path, `/api/receive`, and one param, `parseKey`, which lives in [this app's .env](/apps/ai/.env.example) under `PARSE_KEY`. The full URL should look like `https://abc.ngrok.io/api/receive?parseKey=ABC-123`.
6. Activate "POST the raw, full MIME message".
7. Send an email to `[anyUsername]@example.com`. You should see a ping on the nGrok listener and server.
8. Adjust the logic in [receive/route.ts](/apps/ai/src/app/api/receive/route.ts), save to hot-reload, and send another email to test the behaviour.
Please feel free to improve any part of this architecture.
Please feel free to improve any part of this architecture!

View File

@ -1,6 +1,6 @@
{
"name": "@calcom/ai",
"version": "1.1.1",
"version": "1.2.0",
"private": true,
"author": "Cal.com Inc.",
"dependencies": {
@ -8,7 +8,7 @@
"@t3-oss/env-nextjs": "^0.6.1",
"langchain": "^0.0.131",
"mailparser": "^3.6.5",
"next": "^13.4.6",
"next": "^13.4.7",
"supports-color": "8.1.1",
"zod": "^3.22.2"
},

View File

@ -0,0 +1,44 @@
import type { NextRequest } from "next/server";
import prisma from "@calcom/prisma";
import { env } from "../../../env.mjs";
import sendEmail from "../../../utils/sendEmail";
export const POST = async (request: NextRequest) => {
const { userId } = await request.json();
const user = await prisma.user.findUnique({
select: {
email: true,
name: true,
username: true,
},
where: {
id: userId,
},
});
if (!user) {
return new Response("User not found", { status: 404 });
}
await sendEmail({
subject: "Welcome to Cal AI",
to: user.email,
from: `${user.username}@${env.SENDER_DOMAIN}`,
text: `Hi ${
user.name || `@${user.username}`
},\n\nI'm Cal AI, your personal booking assistant! I'll be here, 24/7 to help manage your busy schedule and find times to meet with the people you care about.\n\nHere are some things you can ask me:\n\n- "Book a meeting with @someone" (The @ symbol lets you tag Cal.com users)\n- "What meetings do I have today?" (I'll show you your schedule)\n- "Find a time for coffee with someone@gmail.com" (I'll intro and send them some good times)\n\nI'm still learning, so if you have any feedback, please tweet it to @calcom!\n\nRemember, you can always reach me here, at ${
user.username
}@${
env.SENDER_DOMAIN
}.\n\nLooking forward to working together (:\n\n- Cal AI, Your personal booking assistant`,
html: `Hi ${
user.name || `@${user.username}`
},<br><br>I'm Cal AI, your personal booking assistant! I'll be here, 24/7 to help manage your busy schedule and find times to meet with the people you care about.<br><br>Here are some things you can ask me:<br><br>- "Book a meeting with @someone" (The @ symbol lets you tag Cal.com users)<br>- "What meetings do I have today?" (I'll show you your schedule)<br>- "Find a time for coffee with someone@gmail.com" (I'll intro and send them some good times)<br><br>I'm still learning, so if you have any feedback, please send it to <a href="https://twitter.com/calcom">@calcom</a> on X!<br><br>Remember, you can always reach me here, at ${
user.username
}@${env.SENDER_DOMAIN}.<br><br>Looking forward to working together (:<br><br>- Cal AI`,
});
return new Response("OK", { status: 200 });
};

View File

@ -59,7 +59,7 @@ export const POST = async (request: NextRequest) => {
},
},
},
where: { email: envelope.from, credentials: { some: { appId: env.APP_ID } } },
where: { email: envelope.from },
});
// User is not a cal.com user or is using an unverified email.

View File

@ -20,6 +20,7 @@ export const env = createEnv({
FRONTEND_URL: process.env.FRONTEND_URL,
APP_ID: process.env.APP_ID,
APP_URL: process.env.APP_URL,
SENDER_DOMAIN: process.env.SENDER_DOMAIN,
PARSE_KEY: process.env.PARSE_KEY,
NODE_ENV: process.env.NODE_ENV,
OPENAI_API_KEY: process.env.OPENAI_API_KEY,
@ -36,6 +37,7 @@ export const env = createEnv({
FRONTEND_URL: z.string().url(),
APP_ID: z.string().min(1),
APP_URL: z.string().url(),
SENDER_DOMAIN: z.string().min(1),
PARSE_KEY: z.string().min(1),
NODE_ENV: z.enum(["development", "test", "production"]),
OPENAI_API_KEY: z.string().min(1),

Binary file not shown.

After

Width:  |  Height:  |  Size: 125 KiB

View File

@ -47,7 +47,7 @@ const createBooking = async ({
}
const responses = {
id: invite,
id: invite.toString(),
name: user.username,
email: user.email,
};

View File

@ -0,0 +1,124 @@
import { DynamicStructuredTool } from "langchain/tools";
import { z } from "zod";
import { env } from "~/src/env.mjs";
import type { User, UserList } from "~/src/types/user";
import sendEmail from "~/src/utils/sendEmail";
export const sendBookingEmail = async ({
user,
agentEmail,
subject,
to,
message,
eventTypeSlug,
slots,
date,
}: {
apiKey: string;
user: User;
users: UserList;
agentEmail: string;
subject: string;
to: string;
message: string;
eventTypeSlug: string;
slots?: {
time: string;
text: string;
}[];
date: {
date: string;
text: string;
};
}) => {
// const url = `${env.FRONTEND_URL}/${user.username}/${eventTypeSlug}?date=${date}`;
const timeUrls = slots?.map(({ time, text }) => {
return {
url: `${env.FRONTEND_URL}/${user.username}/${eventTypeSlug}?slot=${time}`,
text,
};
});
const dateUrl = {
url: `${env.FRONTEND_URL}/${user.username}/${eventTypeSlug}?date=${date.date}`,
text: date.text,
};
await sendEmail({
subject,
to,
cc: user.email,
from: agentEmail,
text: message
.split("[[[Slots]]]")
.join(timeUrls?.map(({ url, text }) => `${text}: ${url}`).join("\n"))
.split("[[[Link]]]")
.join(`${dateUrl.text}: ${dateUrl.url}`),
html: message
.split("\n")
.join("<br>")
.split("[[[Slots]]]")
.join(timeUrls?.map(({ url, text }) => `<a href="${url}">${text}</a>`).join("<br>"))
.split("[[[Link]]]")
.join(`<a href="${dateUrl.url}">${dateUrl.text}</a>`),
});
return "Booking link sent";
};
const sendBookingEmailTool = (apiKey: string, user: User, users: UserList, agentEmail: string) => {
return new DynamicStructuredTool({
description:
"Send a booking link via email. Useful for scheduling with non cal users. Be confident, suggesting a good date/time with a fallback to a link to select a date/time.",
func: async ({ message, subject, to, eventTypeSlug, slots, date }) => {
return JSON.stringify(
await sendBookingEmail({
apiKey,
user,
users,
agentEmail,
subject,
to,
message,
eventTypeSlug,
slots,
date,
})
);
},
name: "sendBookingEmail",
schema: z.object({
message: z
.string()
.describe(
"A polite and professional email with an intro and signature at the end. Specify you are the AI booking assistant of the primary user. Use [[[Slots]]] and a fallback [[[Link]]] to inject good times and 'see all times' into messages"
),
subject: z.string(),
to: z
.string()
.describe("email address to send the booking link to. Primary user is automatically CC'd"),
eventTypeSlug: z.string().describe("the slug of the event type to book"),
slots: z
.array(
z.object({
time: z.string().describe("YYYY-MM-DDTHH:mm in UTC"),
text: z.string().describe("minimum readable label. Ex. 4pm."),
})
)
.optional()
.describe("Time slots the external user can click"),
date: z
.object({
date: z.string().describe("YYYY-MM-DD"),
text: z.string().describe('"See all times" or similar'),
})
.describe(
"A booking link that allows the external user to select a date / time. Should be a fallback to time slots"
),
}),
});
};
export default sendBookingEmailTool;

View File

@ -1,81 +0,0 @@
import { DynamicStructuredTool } from "langchain/tools";
import { z } from "zod";
import { env } from "~/src/env.mjs";
import type { User, UserList } from "~/src/types/user";
import sendEmail from "~/src/utils/sendEmail";
export const sendBookingLink = async ({
user,
agentEmail,
subject,
to,
message,
eventTypeSlug,
date,
}: {
apiKey: string;
user: User;
users: UserList;
agentEmail: string;
subject: string;
to: string[];
message: string;
eventTypeSlug: string;
date: string;
}) => {
const url = `${env.FRONTEND_URL}/${user.username}/${eventTypeSlug}?date=${date}`;
await sendEmail({
subject,
to,
cc: user.email,
from: agentEmail,
text: message.split("[[[Booking Link]]]").join(url),
html: message
.split("\n")
.join("<br>")
.split("[[[Booking Link]]]")
.join(`<a href="${url}">Booking Link</a>`),
});
return "Booking link sent";
};
const sendBookingLinkTool = (apiKey: string, user: User, users: UserList, agentEmail: string) => {
return new DynamicStructuredTool({
description: "Send a booking link via email. Useful for scheduling with non cal users.",
func: async ({ message, subject, to, eventTypeSlug, date }) => {
return JSON.stringify(
await sendBookingLink({
apiKey,
user,
users,
agentEmail,
subject,
to,
message,
eventTypeSlug,
date,
})
);
},
name: "sendBookingLink",
schema: z.object({
message: z
.string()
.describe(
"Make sure to nicely format the message and introduce yourself as the primary user's booking assistant. Make sure to include a spot for the link using: [[[Booking Link]]]"
),
subject: z.string(),
to: z
.array(z.string())
.describe("array of emails to send the booking link to. Primary user is automatically CC'd"),
eventTypeSlug: z.string().describe("the slug of the event type to book"),
date: z.string().describe("the date (yyyy-mm-dd) to suggest for the booking"),
}),
});
};
export default sendBookingLinkTool;

View File

@ -6,7 +6,7 @@ import createBookingIfAvailable from "../tools/createBooking";
import deleteBooking from "../tools/deleteBooking";
import getAvailability from "../tools/getAvailability";
import getBookings from "../tools/getBookings";
import sendBookingLink from "../tools/sendBookingLink";
import sendBookingEmail from "../tools/sendBookingEmail";
import updateBooking from "../tools/updateBooking";
import type { EventType } from "../types/eventType";
import type { User, UserList } from "../types/user";
@ -35,7 +35,7 @@ const agent = async (
createBookingIfAvailable(apiKey, userId, users),
updateBooking(apiKey, userId),
deleteBooking(apiKey),
sendBookingLink(apiKey, user, users, agentEmail),
sendBookingEmail(apiKey, user, users, agentEmail),
];
const model = new ChatOpenAI({
@ -53,6 +53,8 @@ const agent = async (
Make sure your final answers are definitive, complete and well formatted.
Sometimes, tools return errors. In this case, try to handle the error intelligently or ask the user for more information.
Tools will always handle times in UTC, but times sent to users should be formatted per that user's timezone.
In responses to users, always summarize necessary context and open the door to follow ups. For example "I have booked your chat with @username for 3pm on Wednesday, December 20th, 2023 EST. Please let me know if you need to reschedule."
If you can't find a referenced user, ask the user for their email or @username. Make sure to specify that usernames require the @username format. Users don't know other users' userIds.
The primary user's id is: ${userId}
The primary user's username is: ${user.username}

View File

@ -6,8 +6,12 @@ import type { UserList } from "../types/user";
* Extracts usernames (@Example) and emails (hi@example.com) from a string
*/
export const extractUsers = async (text: string) => {
const usernames = text.match(/(?<![a-zA-Z0-9_.])@[a-zA-Z0-9_]+/g)?.map((username) => username.slice(1));
const emails = text.match(/[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+/g);
const usernames = text
.match(/(?<![a-zA-Z0-9_.])@[a-zA-Z0-9_]+/g)
?.map((username) => username.slice(1).toLowerCase());
const emails = text
.match(/[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+/g)
?.map((email) => email.toLowerCase());
const dbUsersFromUsernames = usernames
? await prisma.user.findMany({

View File

@ -24,6 +24,7 @@
"edit-app-template": "yarn app-store edit-template",
"delete-app-template": "yarn app-store delete-template",
"build": "turbo run build --filter=@calcom/web...",
"build:ai": "turbo run build --scope=\"@calcom/ai\"",
"clean": "find . -name node_modules -o -name .next -o -name .turbo -o -name dist -type d -prune | xargs rm -rf",
"db-deploy": "turbo run db-deploy",
"db-seed": "turbo run db-seed",

View File

@ -33,6 +33,19 @@ export async function getHandler(req: NextApiRequest, res: NextApiResponse) {
},
});
await fetch(
`${process.env.NODE_ENV === "development" ? "http://localhost:3005" : "https://cal.ai"}/api/onboard`,
{
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
userId: session.user.id,
}),
}
);
return { url: getInstalledAppPath({ variant: appConfig.variant, slug: "cal-ai" }) };
}

View File

@ -51,6 +51,7 @@
"BACKEND_URL",
"APP_ID",
"APP_URL",
"SENDER_DOMAIN",
"PARSE_KEY",
"NODE_ENV",
"OPENAI_API_KEY",