Skip to main content

Integration OAuth Standards

Yew Search supports OAuth 2.0 integrations using a unified endpoint pattern, similar to Home Assistant. This allows users to securely authorize external services (Gmail, Slack, Google Drive, etc.) without the application ever seeing their passwords.

Overview​

OAuth integrations in Yew Search use a two-step flow:

  1. Authorization Initiation - User clicks "Connect" button, backend generates OAuth URL and redirects to provider
  2. Authorization Callback - Provider redirects back with authorization code, backend exchanges it for tokens and stores them

All OAuth integrations share the same endpoint structure, keeping the core system simple while allowing individual integrations to handle their specific OAuth details.

Endpoints​

Authorization Initiation​

GET /v1/user-integration/oauth/:integration/authorize

Parameters:

  • :integration - Integration domain name (e.g., gmail, slack)

Query Parameters:

  • userId - The authenticated user's ID (from session)

Response:

  • 302 redirect to the OAuth provider's authorization page

Authorization Callback​

GET /v1/user-integration/oauth/:integration/callback

Parameters:

  • :integration - Integration domain name (must match the initiation request)

Query Parameters:

  • code - Authorization code from the OAuth provider
  • state - CSRF protection token (generated during initiation, validated on callback)
  • error - (optional) Error code if user denied or authorization failed

Response:

  • 302 redirect to frontend success/error page
  • Tokens are stored in user_integration table if successful

OAuth Flow​

1. User clicks "Connect Gmail" in UI
   ↓
2. Frontend → GET /v1/user-integration/oauth/gmail/authorize?userId={userId}
   ↓
3. Backend generates state token and stores in session
   ↓
4. Backend calls Integration.generateAuthorizationUrl(redirectUri, state)
   ↓
5. Backend → 302 redirect to Google OAuth (https://accounts.google.com/o/oauth2/v2/auth?...)
   ↓
6. User grants permission on Google
   ↓
7. Google → 302 redirect to /v1/user-integration/oauth/gmail/callback?code=xxx&state=yyy
   ↓
8. Backend validates state token
   ↓
9. Backend calls Integration.exchangeCodeForTokens(code, redirectUri)
   ↓
10. Backend stores credentials in user_integration table
    ↓
11. Backend → 302 redirect to frontend success page
    ↓
12. User sees "Gmail connected successfully"

Integration Requirements​

Integrations that support OAuth must implement three static methods in addition to the standard integration methods:

1. generateAuthorizationUrl()​

Generates the OAuth provider's authorization URL with the correct scopes and parameters.

Signature:

static generateAuthorizationUrl(
  redirectUri: string,
  state: string,
): string

Parameters:

  • redirectUri - The callback URL where the provider will redirect after authorization
  • state - CSRF protection token generated by the backend

Returns:

  • Full authorization URL to redirect the user to

2. exchangeCodeForTokens()​

Exchanges the authorization code for access tokens by calling the OAuth provider's token endpoint.

Signature:

static async exchangeCodeForTokens(
  code: string,
  redirectUri: string,
): Promise<Record<string, any>>

Parameters:

  • code - Authorization code received from the provider
  • redirectUri - The same redirect URI used in the authorization request (required by OAuth spec)

Returns:

  • Object containing the raw token response from the provider (access_token, refresh_token, expires_in, etc.)

3. validateTokens()​

Validates and structures the tokens into the credentials format for storage. This method ensures the tokens contain all required fields.

Signature:

static validateTokens(
  tokens: Record<string, any>,
): CredentialsType

Parameters:

  • tokens - Raw token response from exchangeCodeForTokens()

Returns:

  • Validated credentials object ready for storage in the database

Throws:

  • Error if required fields are missing or invalid

Environment Configuration​

OAuth integrations require environment variables for client credentials:

# Gmail OAuth Configuration
GMAIL_OAUTH_CLIENT_ID=your_client_id.apps.googleusercontent.com
GMAIL_OAUTH_CLIENT_SECRET=your_client_secret

# Slack OAuth Configuration
SLACK_OAUTH_CLIENT_ID=your_slack_client_id
SLACK_OAUTH_CLIENT_SECRET=your_slack_client_secret

# Base URL for OAuth callbacks
OAUTH_REDIRECT_BASE_URL=http://localhost:3000
# Constructs: ${OAUTH_REDIRECT_BASE_URL}/v1/user-integration/oauth/{integration}/callback

Important: In production, OAUTH_REDIRECT_BASE_URL must use HTTPS. OAuth providers reject non-HTTPS redirect URIs for security.

Security Considerations​

State Parameter (CSRF Protection)​

The state parameter prevents Cross-Site Request Forgery attacks:

  1. Backend generates a random string (UUID) during authorization initiation
  2. Backend stores the state in the user's session with a short TTL (5 minutes)
  3. Backend includes state in the OAuth authorization URL
  4. Provider includes the same state in the callback URL
  5. Backend validates the state matches the stored value before processing the callback

If state validation fails, the request is rejected and the user is redirected to an error page.

Token Storage​

All OAuth tokens are stored encrypted in the user_integration table:

  • Tokens are encrypted at rest using application encryption keys
  • Database administrators cannot read user tokens
  • Tokens are only decrypted when needed for API calls

HTTPS Requirement​

OAuth providers require HTTPS for redirect URIs in production. During local development, providers usually allow http://localhost as an exception.

Error Handling​

OAuth can fail at multiple points:

Authorization Denied:

  • User clicks "Cancel" on the OAuth consent screen
  • Provider redirects to callback with ?error=access_denied
  • Backend redirects to frontend with error message
  • User sees: "Gmail authorization was cancelled"

Invalid State:

  • State parameter doesn't match stored value (potential CSRF attack)
  • Backend rejects request immediately
  • User sees: "Authorization failed - invalid state token"

Token Exchange Failed:

  • Authorization code is invalid or expired
  • Provider returns error from token endpoint
  • Backend logs error and redirects to frontend error page
  • User sees: "Failed to complete Gmail authorization"

Integration Not Found:

  • User requests OAuth for non-existent integration
  • Backend returns 404
  • User sees: "Integration not found"

Missing Credentials:

  • Environment variables for client ID/secret are not set
  • Backend throws error during authorization URL generation
  • Admin sees: "GMAIL_OAUTH_CLIENT_ID not configured"

Complete Integration Example​

Here's a complete Gmail OAuth integration with all required methods (signatures only):

import { google, gmail_v1 } from 'googleapis';
import {
  BaseIntegration,
  BaseAuthCredentials,
  BaseTask,
  BaseTaskOutput,
  BaseTaskResult,
  ContentExistenceChecker,
} from '../../_base/main';

// ============================================================================
// Credentials
// ============================================================================

export class GmailAuthCredentials extends BaseAuthCredentials {
  clientId: string;
  clientSecret: string;
  accessToken: string;
  refreshToken: string;
  expiresAt: Date;

  constructor(credentials: GmailAuthCredentials) {
    super();
    this.clientId = credentials.clientId;
    this.clientSecret = credentials.clientSecret;
    this.accessToken = credentials.accessToken;
    this.refreshToken = credentials.refreshToken;
    this.expiresAt = credentials.expiresAt;
  }
}

// ============================================================================
// Task Types
// ============================================================================

export type GmailTaskType = 'start' | 'getEmailList' | 'downloadEmail';

export class GmailTaskPayload {
  emailId?: string | null;
  pageToken?: string | null;

  constructor(config: GmailTaskPayload) {
    this.emailId = config.emailId;
    this.pageToken = config.pageToken;
  }
}

export class GmailTask extends BaseTask {
  type: GmailTaskType;
  payload: GmailTaskPayload;

  constructor(type: GmailTaskType, payload: GmailTaskPayload) {
    super({ type, payload });
    this.type = type;
    this.payload = payload;
  }
}

export class GmailTaskOutput extends BaseTaskOutput {}

export class GmailTaskResult extends BaseTaskResult {
  output: GmailTaskOutput[];
  tasks?: GmailTask[];

  constructor(config?: GmailTaskResult) {
    super();
    this.output = config?.output || [];
    this.tasks = config?.tasks || [];
  }
}

// ============================================================================
// Integration Class
// ============================================================================

export class GmailIntegration extends BaseIntegration {
  private oAuth2Client: any;
  private gmailClient: gmail_v1.Gmail;

  // --------------------------------------------------------------------------
  // OAuth Static Methods (NEW - Required for OAuth support)
  // --------------------------------------------------------------------------

  /**
   * Generates the OAuth authorization URL for Gmail
   */
  static generateAuthorizationUrl(
    redirectUri: string,
    state: string,
  ): string {
    // Implementation omitted
  }

  /**
   * Exchanges authorization code for access and refresh tokens
   */
  static async exchangeCodeForTokens(
    code: string,
    redirectUri: string,
  ): Promise<Record<string, any>> {
    // Implementation omitted
  }

  /**
   * Validates and structures tokens for storage
   */
  static validateTokens(
    tokens: Record<string, any>,
  ): GmailAuthCredentials {
    // Implementation omitted
  }

  // --------------------------------------------------------------------------
  // Standard Integration Methods
  // --------------------------------------------------------------------------

  constructor(
    credentials: GmailAuthCredentials,
    contentExists?: ContentExistenceChecker,
  ) {
    super(credentials, contentExists);
    // Setup OAuth2 client and Gmail API client
  }

  /**
   * Initialize connection and refresh tokens if needed
   */
  public async initialize(): Promise<void> {
    // Implementation omitted
  }

  /**
   * Validate that the stored credentials still work
   */
  public async validateAuthentication(): Promise<boolean> {
    // Implementation omitted
  }

  /**
   * Main task router
   */
  public async runTask(task: GmailTask): Promise<GmailTaskResult> {
    // Implementation omitted
  }

  /**
   * Clean up connections
   */
  public async shutdown(): Promise<void> {
    // Implementation omitted
  }

  // --------------------------------------------------------------------------
  // Task Handlers (Private)
  // --------------------------------------------------------------------------

  private async handleGetEmailList(
    task: GmailTask,
  ): Promise<GmailTaskResult> {
    // Implementation omitted
  }

  private async handleDownloadEmail(
    task: GmailTask,
  ): Promise<GmailTaskResult> {
    // Implementation omitted
  }

  // --------------------------------------------------------------------------
  // Helper Methods (Private)
  // --------------------------------------------------------------------------

  private extractSubject(message: gmail_v1.Schema$Message): string | null {
    // Implementation omitted
  }

  private extractFrom(message: gmail_v1.Schema$Message): string | null {
    // Implementation omitted
  }

  private extractBody(message: gmail_v1.Schema$Message): string {
    // Implementation omitted
  }
}

// ============================================================================
// Integration Export
// ============================================================================

export const integration = () => {
  return {
    manifest: require('../manifest.json'),
    Integration: GmailIntegration,
  };
};

OAuth vs Non-OAuth Integrations​

Not all integrations need OAuth. Some use simpler authentication methods:

OAuth Integrations:

  • Gmail (Google OAuth)
  • Slack (Slack OAuth)
  • Google Drive (Google OAuth)
  • Microsoft 365 (Microsoft OAuth)

Non-OAuth Integrations:

  • FTP/SFTP (username + password + host)
  • IMAP (username + password + host)
  • RSS feeds (often no authentication)
  • Webhook receivers (API key or no auth)

For non-OAuth integrations, simply omit the three static OAuth methods. Users will configure credentials through a different flow (manual form input, for example).

Backend Implementation Notes​

The user-integration/oauth controller will:

  1. Load the integration class dynamically using the integration loader
  2. Call the static OAuth methods without instantiating the integration
  3. Store the validated credentials in the user_integration table
  4. Never execute tasks during OAuth flow - tasks run separately via the polling system

This keeps OAuth authorization separate from task execution, maintaining clean separation of concerns.

Testing OAuth Integrations​

When developing OAuth integrations:

  1. Register OAuth application with the provider (Google, Slack, etc.)
  2. Set redirect URI to http://localhost:3000/v1/user-integration/oauth/{integration}/callback
  3. Configure environment variables with client ID and secret
  4. Test authorization flow in browser
  5. Verify tokens are stored correctly in database
  6. Test token refresh by simulating expired tokens
  7. Test error cases (user denies, invalid code, etc.)

Next Steps​

After implementing OAuth support in your integration:

  1. Update manifest.json to indicate OAuth support (optional field for future UI)
  2. Test the full authorization flow end-to-end
  3. Document any provider-specific quirks or requirements
  4. Add frontend UI for the "Connect" button and success/error pages