Email body
', - // Threading headers work the same - inReplyTo: 'Reply message
', - inReplyTo: 'This is HTML
', - text: 'Hello\n\nThis is plain text', -}); -``` - -### ✅ CC, BCC, Reply-To - -All standard email fields work: - -```typescript -await ctx.runMutation(api.conversations.sendMessage, { - // ... other fields - to: ['recipient@example.com'], - cc: ['cc@example.com'], - bcc: ['bcc@example.com'], - replyTo: 'reply-to@example.com', -}); -``` - -## Migration from SMTP to API - -### Option 1: Update Existing Provider - -```typescript -// Update an existing provider to use API -await ctx.runMutation(api.email_providers.update, { - providerId: 'existing-provider-id', - sendMethod: 'api', // Switch from SMTP to API - // Remove smtpConfig if no longer needed - smtpConfig: undefined, -}); -``` - -### Option 2: Create New Provider - -Create a new provider with `sendMethod: 'api'` and set it as default. - -## Troubleshooting - -### Error: "API sending requires OAuth2 authentication" - -**Solution:** API sending only works with OAuth2. Password authentication is not supported for API sending. - -### Error: "Unsupported provider for API sending" - -**Solution:** Only Gmail and Microsoft providers support API sending. For other providers, use SMTP. - -### Error: "Gmail API error: 401 Unauthorized" - -**Solution:** OAuth2 token expired. The system should automatically refresh it. If not, re-authorize the provider. - -### Error: "Microsoft Graph API error: 403 Forbidden" - -**Solution:** Check that the OAuth2 app has the required scopes (`Mail.Send`). - -## Comparison: SMTP vs API - -| Feature | SMTP | API | -|---------|------|-----| -| **Port blocking** | ❌ Often blocked | ✅ Never blocked (HTTPS) | -| **DNS setup** | ⚠️ Required for custom domains | ✅ Not needed for Gmail/M365 | -| **OAuth2 support** | ✅ Yes (XOAUTH2) | ✅ Yes (native) | -| **Password auth** | ✅ Yes | ❌ No | -| **Threading** | ✅ Yes | ✅ Yes | -| **Message ID** | ✅ Yes | ✅ Yes | -| **Attachments** | ✅ Yes | ⚠️ Not yet implemented | -| **Rate limits** | ⚠️ Provider-specific | ✅ Higher limits | - -## Best Practices - -1. **Use API for cloud deployments** - Avoid port blocking issues -2. **Use SMTP for on-premise** - If you control the network -3. **Keep OAuth2 tokens fresh** - System handles this automatically -4. **Monitor token expiry** - Check provider status regularly -5. **Test before production** - Send test emails to verify configuration - -## Example: Complete Setup - -```typescript -// 1. Create OAuth2 provider with API sending -const providerId = await ctx.runAction(api.email_providers.createOAuth2Provider, { - organizationId: 'org-123', - name: 'Company Outlook', - vendor: 'outlook', - sendMethod: 'api', - oauth2Auth: { - provider: 'microsoft', - clientId: process.env.MICROSOFT_CLIENT_ID, - clientSecret: process.env.MICROSOFT_CLIENT_SECRET, - authorizationCode: authCode, - accountType: 'organizational', - }, - imapConfig: { - host: 'outlook.office365.com', - port: 993, - secure: true, - }, -}); - -// 2. Send email (automatically uses API) -const messageId = await ctx.runMutation(api.conversations.sendMessage, { - conversationId: conversationId, - organizationId: 'org-123', - providerId: providerId, - content: 'Hello from API!', - to: ['customer@example.com'], - subject: 'Welcome', - html: 'Thanks for signing up.
', -}); - -// 3. Message is sent via Microsoft Graph API -// 4. Internet Message ID is stored for threading -``` - -## Next Steps - -- [ ] Add attachment support for API sending -- [ ] Add delivery status tracking via webhooks -- [ ] Add read receipt support -- [ ] Add batch sending optimization - diff --git a/docs/email-providers.md b/docs/email-providers.md deleted file mode 100644 index 35da1a1ee5..0000000000 --- a/docs/email-providers.md +++ /dev/null @@ -1,659 +0,0 @@ -# Email Providers – Convex Integration Guide - -A single, concise guide for developers to understand and integrate the email providers feature using Convex. English-only. Follows Convex best practices. - -## What you need to know - -- Providers supported: Resend (fallback), SMTP (send), IMAP (receive), OAuth2 (Gmail, Microsoft), Password auth -- If both SMTP and IMAP are configured, both are validated and used -- Local dev uses Inbucket (see Local Development Setup section below) -- All data stored in Convex with proper encryption for secrets -- Uses Convex mutations for writes and queries for reads -- **Automatic default**: When creating the first email provider for an organization, it will automatically be set as the default provider, even if `isDefault: false` is specified - -## Convex Schema - -The `emailProviders` table is defined in `convex/schema.ts`: - -```typescript -emailProviders: defineTable({ - organizationId: v.id('organizations'), - name: v.string(), - vendor: v.union( - v.literal('gmail'), - v.literal('outlook'), - v.literal('smtp'), - v.literal('resend'), - v.literal('other'), - ), - authMethod: v.union( - v.literal('password'), - v.literal('oauth2'), - ), - - // Auth configurations (encrypted in metadata) - passwordAuth: v.optional(v.object({ - user: v.string(), - passEncrypted: v.string(), // encrypted password - })), - - oauth2Auth: v.optional(v.object({ - provider: v.string(), // 'gmail' | 'microsoft' - clientId: v.string(), - clientSecretEncrypted: v.string(), // encrypted secret - accessTokenEncrypted: v.optional(v.string()), - refreshTokenEncrypted: v.optional(v.string()), - tokenExpiry: v.optional(v.number()), - })), - - // SMTP configuration - smtpConfig: v.optional(v.object({ - host: v.string(), - port: v.number(), - secure: v.boolean(), - })), - - // IMAP configuration - imapConfig: v.optional(v.object({ - host: v.string(), - port: v.number(), - secure: v.boolean(), - })), - - // Status and metadata - isActive: v.boolean(), - isDefault: v.boolean(), - status: v.optional(v.union( - v.literal('active'), - v.literal('error'), - v.literal('testing'), - )), - lastTestedAt: v.optional(v.number()), - lastSyncAt: v.optional(v.number()), - errorMessage: v.optional(v.string()), - - metadata: v.optional(v.any()), -}) - .index('by_organizationId', ['organizationId']) - .index('by_organizationId_and_vendor', ['organizationId', 'vendor']) - .index('by_organizationId_and_isDefault', ['organizationId', 'isDefault']) - .index('by_organizationId_and_status', ['organizationId', 'status']), -``` - -## UI flows (recommended) - -1. Providers list page (simplified) - -- Show table of providers (name, vendor, status, default, last tested) -- Buttons: Create, Set Default, Test, Edit, Delete - -2. Create provider page (dedicated) - -- Tabs are not required; use a single page with radio for Auth Method: Password or OAuth2 -- Show vendor presets (Gmail/Outlook) to prefill hostnames/ports -- If OAuth2 + Microsoft (use existing login) is selected, we redirect to consent - -3. Test provider action - -- Call mutation to test; show SMTP and IMAP results separately - -4. Receive email sync - -- Manual "Sync now" triggers action; show results (processed/total/errors) - -## Convex Functions (import paths) - -All functions are in `convex/emailProviders.ts`: - -### Queries - -```typescript -import { api } from '@/convex/_generated/api'; - -// List all providers for an organization -api.emailProviders.list; - -// Get single provider by ID -api.emailProviders.get; - -// Get default provider for an organization -api.emailProviders.getDefault; -``` - -### Mutations - -```typescript -// Update provider configuration -api.emailProviders.update; - -// Delete provider -api.emailProviders.delete; - -// Set provider as default -api.emailProviders.setDefault; -``` - -### Actions - -```typescript -// Create email provider (password or OAuth2) - with encryption -api.emailProviders.create; - -// Test SMTP/IMAP connectivity -api.emailProviders.test; - -// Send email using provider -api.emailProviders.sendEmail; - -// Sync received emails via IMAP -api.emailProviders.syncEmails; - -// Initialize OAuth2 flow (returns auth URL) -api.emailProviders.initOAuth2; - -// Handle OAuth2 callback -api.emailProviders.handleOAuth2Callback; -``` - -## Quick start – examples - -### Using from React components - -```typescript -'use client'; - -import { useQuery, useMutation, useAction } from 'convex/react'; -import { api } from '@/convex/_generated/api'; -import type { Id } from '@/convex/_generated/dataModel'; - -function EmailProvidersPage({ organizationId }: { organizationId: Id<'organizations'> }) { - // List providers - const providers = useQuery(api.emailProviders.list, { organizationId }); - - // Create provider action (handles encryption) - const createProvider = useAction(api.emailProviders.create); - - // Handle create - const handleCreate = async () => { - await createProvider({ - organizationId, - name: 'Gmail (App Password)', - vendor: 'gmail', - authMethod: 'password', - passwordAuth: { - user: 'you@gmail.com', - pass: 'app-password' // will be encrypted - }, - smtpConfig: { host: 'smtp.gmail.com', port: 587, secure: false }, - imapConfig: { host: 'imap.gmail.com', port: 993, secure: true }, - isActive: true, - isDefault: true, - }); - }; - - return ( -Hi
', - // Optional: specify provider, otherwise uses default - providerId: myProviderId, -}); -``` - -### Sync received emails - -```typescript -import { useAction } from 'convex/react'; -import { api } from '@/convex/_generated/api'; - -const syncEmails = useAction(api.emailProviders.syncEmails); - -const result = await syncEmails({ - organizationId, - limit: 50, -}); -console.log(`Synced ${result.processed} of ${result.total} emails`); -``` - -## OAuth2 Integration - -### OAuth2 Flow - -1. Call `api.emailProviders.initOAuth2` action to start the flow -2. User is redirected to provider consent screen -3. After consent, user is redirected back to `/api/auth/oauth2/callback` -4. Callback handler calls `api.emailProviders.handleOAuth2Callback` to complete setup -5. Tokens are encrypted and stored in the `emailProviders` table - -### Redirect Configuration - -- OAuth2 callback route: `/api/auth/oauth2/callback` -- Configure this exact URL in your OAuth2 provider console (Gmail/Microsoft) -- Ensure `SITE_URL` is set correctly in `.env` (or `DOMAIN` in Docker Compose) - -### Required Environment Variables - -```bash -# Base URL for OAuth2 callbacks (set via DOMAIN in Docker Compose) -SITE_URL=http://localhost:3000 - -# For Microsoft existing account flow (optional) -AUTH_MICROSOFT_ENTRA_ID_ID=your-client-id -AUTH_MICROSOFT_ENTRA_ID_SECRET=your-client-secret -AUTH_MICROSOFT_ENTRA_ID_ISSUER=https://login.microsoftonline.com/common/v2.0 -``` - -### Scopes and Prerequisites - -**Gmail** - -- Scope: `https://mail.google.com/` (covers SMTP/IMAP/POP via XOAUTH2) -- You do NOT need `gmail.send` or `gmail.readonly` unless you call the Gmail REST API -- We request refresh tokens with `access_type=offline` and `prompt=consent` -- Enable IMAP in Gmail settings - -**Microsoft 365 / Outlook** - -- Scopes: `https://outlook.office.com/SMTP.Send`, `https://outlook.office.com/IMAP.AccessAsUser.All`, `offline_access` -- Ensure IMAP and SMTP AUTH are enabled in the tenant and for the mailbox in Exchange Online -- Choose tenant endpoint based on account type: consumers, organizations, or common - -### OAuth2 Token Management - -- Tokens are encrypted using Convex encryption utilities before storage -- Stored in the `oauth2Auth` field of the `emailProviders` table -- Automatic token refresh is handled by scheduled Convex cron jobs -- Token expiry is tracked in the `tokenExpiry` field (Unix timestamp) - -### Implementation Details - -**Convex Files** - -- `convex/emailProviders.ts` – main queries, mutations, and actions -- `convex/lib/email-oauth2.ts` – OAuth2 utilities (token exchange, refresh, encryption) -- `convex/crons.ts` – scheduled token refresh job -- `app/api/auth/oauth2/callback/route.ts` – Next.js route handler for OAuth2 callbacks - -**Token Refresh Strategy** - -- Cron job runs every 30 minutes -- Checks for tokens expiring in the next hour -- Refreshes tokens proactively before expiry -- Updates `tokenExpiry` and `accessTokenEncrypted` fields - -## Convex Best Practices for Email Providers - -### Query Patterns - -```typescript -// ✅ Good: Use indexed query for performance -const providers = await ctx.db - .query('emailProviders') - .withIndex('by_organizationId', (q) => q.eq('organizationId', organizationId)) - .filter((q) => q.eq(q.field('isActive'), true)) - .collect(); - -// ❌ Bad: Don't use filter without index -const providers = await ctx.db - .query('emailProviders') - .filter((q) => q.eq(q.field('organizationId'), organizationId)) - .collect(); -``` - -### Mutation Patterns - -```typescript -// ✅ Good: Validate and encrypt secrets -export const create = mutation({ - args: { - organizationId: v.id('organizations'), - name: v.string(), - passwordAuth: v.optional( - v.object({ - user: v.string(), - pass: v.string(), // will be encrypted - }), - ), - // ... other args - }, - handler: async (ctx, args) => { - // Encrypt password before storage - const passEncrypted = args.passwordAuth?.pass - ? await encrypt(args.passwordAuth.pass) - : undefined; - - return await ctx.db.insert('emailProviders', { - ...args, - passwordAuth: passEncrypted - ? { - user: args.passwordAuth!.user, - passEncrypted, - } - : undefined, - }); - }, -}); -``` - -### Action Patterns - -```typescript -// ✅ Good: Use actions for external API calls (SMTP/IMAP) -export const test = action({ - args: { providerId: v.id('emailProviders') }, - returns: v.object({ - success: v.boolean(), - smtp: v.optional( - v.object({ - success: v.boolean(), - latencyMs: v.number(), - error: v.optional(v.string()), - }), - ), - imap: v.optional( - v.object({ - success: v.boolean(), - latencyMs: v.number(), - error: v.optional(v.string()), - }), - ), - }), - handler: async (ctx, args) => { - // Get provider data via query - const provider = await ctx.runQuery(internal.emailProviders.getInternal, { - providerId: args.providerId, - }); - - // Test SMTP/IMAP connections (external API calls) - const smtpResult = await testSmtpConnection(provider); - const imapResult = await testImapConnection(provider); - - // Update provider status via mutation - await ctx.runMutation(internal.emailProviders.updateStatus, { - providerId: args.providerId, - status: smtpResult.success && imapResult.success ? 'active' : 'error', - lastTestedAt: Date.now(), - }); - - return { - success: smtpResult.success && imapResult.success, - smtp: smtpResult, - imap: imapResult, - }; - }, -}); -``` - -## Troubleshooting - -### Common Issues - -**Gmail password auth fails** - -- Ensure App Passwords are enabled (requires 2FA) -- Consider migrating to OAuth2 for better security -- Verify IMAP is enabled in Gmail settings - -**OAuth2 authorization errors** - -- Verify scopes match exactly in provider console -- Ensure redirect URI matches exactly (including protocol and port) -- Check that `SITE_URL` is configured correctly (or `DOMAIN` in Docker Compose) - -**Missing refresh tokens** - -- Gmail: Ensure `access_type=offline` and `prompt=consent` are set -- Microsoft: Ensure `offline_access` scope is included - -**SMTP/IMAP connection errors** - -- Gmail: `smtp.gmail.com:587` (SMTP), `imap.gmail.com:993` (IMAP) -- Outlook: `smtp.office365.com:587` (SMTP), `outlook.office365.com:993` (IMAP) -- Verify firewall rules and network connectivity -- Check provider rate limits - -**Slow SMTP performance** - -- Increase connection/greeting/socket timeouts -- Consider connection pooling for high-volume scenarios - -**TLS issues (development only)** - -- For local dev, you may set `tls.rejectUnauthorized=false` -- NEVER use this in production - -### Migration from Password to OAuth2 - -1. Create a new OAuth2 provider for the same mailbox -2. Complete OAuth2 consent flow -3. Test the new provider -4. Set new provider as default: `api.emailProviders.setDefault` -5. Deactivate the old password-based provider -6. Optionally delete the old provider after confirming the new one works - -## UI Implementation Guidelines - -### Provider List Component - -```typescript -'use client'; - -import { useQuery, useMutation, useAction } from 'convex/react'; -import { api } from '@/convex/_generated/api'; -import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from '@/components/ui/table'; -import { Button } from '@/components/ui/button'; -import { Badge } from '@/components/ui/badge'; - -export function EmailProvidersList({ organizationId }) { - const providers = useQuery(api.emailProviders.list, { organizationId }); - const deleteProvider = useMutation(api.emailProviders.delete); - const setDefault = useMutation(api.emailProviders.setDefault); - const testProvider = useAction(api.emailProviders.test); - - return ( -Status: {config.last_sync_status}
-Last sync: {config.last_sync_at}
- - -