DWD with calendar-cache support

apps/api/v1/pages/api/credential-sync/_post.ts

@@ -108,7 +108,7 @@ async function handler(req: NextApiRequest) {
   // ^ Workaround for the select in `create` not working
 
   if (createCalendarResources) {
-    const calendar = await getCalendar(credential);
+    const calendar = await getCalendar({ ...credential, delegatedTo: null });
     if (!calendar) throw new HttpError({ message: "Calendar missing for credential", statusCode: 500 });
     const calendars = await calendar.listCalendars();
     const calendarToCreate = calendars.find((calendar) => calendar.primary) || calendars[0];

apps/api/v1/pages/api/destination-calendars/[id]/_patch.ts

@@ -80,6 +80,7 @@ type UserCredentialType = {
   teamId: number | null;
   key: Prisma.JsonValue;
   invalid: boolean | null;
+  domainWideDelegationCredentialId?: string | null;
 };
 
 export async function patchHandler(req: NextApiRequest) {
@@ -185,7 +186,7 @@ async function verifyCredentialsAndGetId({
   currentCredentialId: number | null;
 }) {
   if (parsedBody.integration && parsedBody.externalId) {
-    const calendarCredentials = getCalendarCredentials(userCredentials);
+    const calendarCredentials = await getCalendarCredentials(userCredentials);
 
     const { connectedCalendars } = await getConnectedCalendars(
       calendarCredentials,

apps/api/v1/pages/api/destination-calendars/_post.ts

@@ -82,7 +82,7 @@ async function postHandler(req: NextApiRequest) {
       message: "Bad request, credential id invalid",
     });
 
-  const calendarCredentials = getCalendarCredentials(userCredentials);
+  const calendarCredentials = await getCalendarCredentials(userCredentials);
 
   const { connectedCalendars } = await getConnectedCalendars(calendarCredentials, [], parsedBody.externalId);
 

apps/web/app/settings/(settings-layout)/organizations/domain-wide-delegation/domain-wide-delegation-TODO.md

@@ -0,0 +1,74 @@
+## Version 1.0
+### Release Plan
+  - Read the document(domain-wide-delegation.md) and acknowledge it.
+  - Deploy
+    1. Follow "Setting up Domain-Wide Delegation for Google Calendar API" in domain-wide-delegation.md to create Service Account and create a workspace.
+    2. Merge PR and then deploy.
+ - Enabling for i.cal.com
+    - 1. Enable DWD for i.cal.com first and then test there
+    - 2. Wait for 1-2 days and keep monitoring the errors in Sentry and Axiom.
+ - Enable for a big customer
+    - 1. Wait for a week and keep monitoring the errors in Sentry and Axiom.
+- Followup with sorting the credentials with DWD credentials first
+- Monitor the errors in Sentry and Axiom.
+
+### Important
+  - Bugs
+    - [ ] Duplicate Calendar Events in Google Calendar when choosing non-primary calendar as destination. No idea why this is happening.
+    - [x] Duplicate Calendar connections in 'apps/installed' if a user already had connected calendar and DWD is enabled.
+    - [x] Calendar Cache has credentialId column which isn't applicable for DWD(Solution: Added userId there)
+    - Manual Testing
+      - [ ] Test with Multiple DWD entries for different organizations. Verify that wrong DWD entry isn't used.
+      - [ ] Location Change of a booking  to Google Meet(from Cal Video)
+      - [ ] RR Team Event
+        - Booking
+          - Unavailable slot isn't available for booking. Unavailable user isn't used.
+        - Reroute
+        - Reassign
+      - [ ] Calendar Cache
+    - [x] Troubleshooter
+      - [ ] Shows busy times from Claendar
+    - [x] If a user has connected a calendar, and then DWD is enabled.
+      - Tested various scenarios for it
+    - [x] Inviting a new user. 
+      - Verified that Google Calendar is shown pre-installed. 
+      - How about Google Meet(which depends on Google Calendar) - Correctly shows up as installed.
+  - TODO:
+    - [x] Troubleshooter
+    - [x] Google CalendarService unit tests to verify that if DWD credential is provided it uses impersonation to access API otherwise it uses regular user credential API.
+    - [x] setDestinationCalendar.handler.ts tests to verify that when DWD is enabled it still correctly sets the destination calendar. 
+    - [x] getConnectedDestinationCalendars tests.
+  - [x] Creating DWD shouldn't immediately enable it. Enabling has separate check to confirm if it is actually configured in google workspace
+  - [x] Added check to avoid adding same domain for a workspace platform in another organization if it is already enabled in some other organization
+  - [x] Don't show dwd in menu for non-org-admin users - It errors with something_went_wrong right now
+  - [x] Don't allow disabled platform to be selected in the UI for creation.
+    - We have disabled coming the disabled platform to be coming into the list that effectively disables edit of existing dwd and creation of new dwd for that platform.
+  - [x] Where should we show the user the client ID to enable domain wide delegation?
+    - [x] It must be shown to the organization owner/admin only
+    - [x] There could be multiple checkboxes per domain to enable domain wide delegation for a domain
+  - [x] Which domain to allow
+    - Any domain can be added by a user
+  - [x] Support multiple domains in DomainWideDelegation schema for an organization
+    - [x] Use the domain as well to identify if the domain wide delegation is enabled
+  - [x] Before enabling Domain-wide delegation, there should be a check to ensure that the clientID has been added to the Workspace Platform
+  - [x] We should allow setting default conferencing app during onboarding
+
+### Follow-up release
+  - [ ] Confirmation for DwD deletion and disabling
+  - [ ] If DWD is enabled and the org member doesn't exist in Google Workspace, and the user has connected personal account, should we correctly use the personal account?
+
+### Security
+  - [x] We don't let any one user see the added service account key from UI.
+  - [ ] We intend to implement Workload Identity Federation in the future.
+
+### Documentation
+- After enabling domain-wide delegation, the credential is shown pre-installed and the connection can't be removed(or the app can't be uninstalled by user)
+- Steps
+  - App admin will first create a Workspace Platform and then organization owner/admin can enable domain-wide delegation for a domain
+  - As soon as domain-wide delegation is created, it would start taking preference over the personal credentials of the organization members and it would be used for that. 
+
+Version-2.0
+- Workload Identity Federation to ensure that the service account key is never stored in DB.
+
+
+

apps/web/app/settings/(settings-layout)/organizations/domain-wide-delegation/domain-wide-delegation.md

@@ -0,0 +1,87 @@
+## Setting up Domain-Wide Delegation for Google Calendar API
+
+Step 1: Create a Google Cloud Project
+
+Before you can create a service account, you'll need to set up a Google Cloud project.
+
+ 1. Create a Google Cloud Project:
+        - Go to the Google Cloud Console
+        - Select Create Project
+        - Give your project a name and select your billing account (if applicable)
+        - Click Create
+  2. Enable the Google Calendar API:
+     1. Go to the Google Cloud Console
+     2. Select API & Services → Library
+     3. Search for "Google Calendar API"
+     4. Click Enable
+
+Step 2: Create a Service Account
+
+A service account is needed to act on behalf of users
+
+ 1. Navigate to the Service Accounts page:
+    - In the Google Cloud Console, go to IAM & Admin → Service Accounts
+ 2. Create a New Service Account:
+    - Click on Create Service Account
+    - Give your service account a name and description
+    - Click Create and Continue
+
+Step 3: To Be taken by Cal.com instance admin:
+   - Create a Workspace Platform with slug="google". Slug has to be exactly this. This is how we know we need to use Google Calendar and Google Meet.
+
+Last Step (To Be Taken By Cal.com organization Owner/Admin): Assign Specific API Permissions via OAuth Scopes:
+    - Create DWD with workspace platform "google"
+      - User must be a member of the Google Workspace to be able to enable DWD as there is a validation if the user's calendar can be accessed through the service account
+    - Get the Client ID from there
+    - Go to your Google Admin Console (admin-google-com)
+    - Navigate to Security → Access and Data Controls -> API controls -> Manage Domain-Wide Delegation
+    - Here, you'll authorize the Client ID(Unique ID) to access the Google Calendar API
+    - Add the necessary API scopes for Google Calendar(Full access to Google Calendar)
+        https://www.googleapis.com/auth/calendar
+
+
+## Restrictions after enabling DWD
+- Enabling DWD for a particular workspace in Cal.com(only google supported at the moment) disables the user from disconnecting that credential.
+
+## Developer Notes
+### How DWD works
+- We use the Cal.com user's email to impersonate that user using DWD Credential(which is just a service account key at the moment)
+   - That gives us read/write permission to get availability of the user and create new events in their calendar.
+
+### What is a DWD Credential?
+- A DWD service account key along with user's email becomes the DWD Credential which is an alternative to regular Credential in DB.
+- DWD doesn't completely replace the regular credentials. DWD Credential gives access to the cal.com user's email in Google Calendar. So, if the user needs to connect to some other email's calendar, we need to use the regular credentials.
+
+### Important Points
+- No Credential table entry is created when enabling DWD. The workspace platform's related apps will be considered as "installed" for the users with email matching dwd domain. An in-memory credential like object is created for this purpose. It allows avoiding creation of thousands of records for all the members of the organization when dwd is enabled.
+- DWD Credential is applicable to Users only. 
+   - For team, we don't use dwd credential as you can impersonate a user and not team through Dwd credential. Currently supported apps(Google Calendar and Google Meet) don't support team installation, so we could simply allow enabling DWD without any issues.
+- Disabling a workspace platform stops it from being used for any new organizations and also disables any DWD using the workspace platform from being edited.
+   - It still all existing DWDs to keep on working
+- Adding any number of DWDs for a particular workspace always gives the same Client ID as DWD uses the workspace's default Service Account.
+- We should disable DWD and not delete it when we want to stop using it temporarily. Deleting DWD also removes all the seletedCalendar entries connected to it. 
+
+### How apps/installed loads the credentials
+1. Identify the logged in user's email
+2. Identify the domainWideDelegations for that email's domain
+3. Build in-memory credentials for the domainWideDelegations and use them along with the actual credentials(that user might have connected) of the user
+4. We don't show the non-dwd connected calendar(if there is a corresponding dwd connected calendar). Though we use the non-dwd credentials to identify the selected calendars, for the dwd connected calendar.
+
+
+## Impact on existing users booking flow
+- There should be no impact on availability on enabling DWD because we keep on using the existing credentials along with new DWD credential.
+- When booking the event, we sort the credentials with DWD credentials last, so there should be no impact on creating calendar events.
+   - NOTE: We will followup with sorting the credentials with DWD credentials first in a followup PR(They are preferred because they don't expire)
+
+## Impact on APIs - [ To Verify ]
+- We don't support DWD through APIs yet. So, all existing APIs would continue to work with non-dwd credentials only.
+
+## Performance Issues
+- There could be 100s of users in an organization with already connected calendars. Enabling DWD adds a duplicate credential for each of them.
+   - Because a credential isn't aware of which email it is for(without connecting with Google Calendar API itself), we can't deduplicate them.
+
+## Notes when testing locally
+- You need to enable the feature through feature flag.
+- You could use Acme org and login as owner1-acme@example.com
+- Make sure to change the email of the user above to your workspace owner's email(other member's email might also work). This is necessary otherwise you won't be able to enable DWD for the organization.
+   - Note: After changing the email, you would have to logout and login again as required by NextAuth

apps/web/components/apps/AppPage.tsx

@@ -79,12 +79,14 @@ export const AppPage = ({
   const searchParams = useCompatSearchParams();
 
   const hasDescriptionItems = descriptionItems && descriptionItems.length > 0;
+  const utils = trpc.useUtils();
 
   const mutation = useAddAppMutation(null, {
-    onSuccess: (data) => {
+    onSuccess: async (data) => {
       if (data?.setupPending) return;
       setIsLoading(false);
-      showToast(t("app_successfully_installed"), "success");
+      showToast(data?.message || t("app_successfully_installed"), "success");
+      await utils.viewer.appCredentialsByType.invalidate({ appType: type });
     },
     onError: (error) => {
       if (error instanceof Error) showToast(error.message || t("app_could_not_be_installed"), "error");
@@ -161,6 +163,7 @@ export const AppPage = ({
 
   // variant not other allows, an app to be shown in calendar category without requiring an actual calendar connection e.g. vimcal
   // Such apps, can only be installed once.
+
   const allowedMultipleInstalls = categories.indexOf("calendar") > -1 && variant !== "other";
   useEffect(() => {
     if (searchParams?.get("defaultInstall") === "true") {