Troubleshooting Authentication & Token Failures

Home > Troubleshooting & RCA Overview > Auth & Token Failures

1. Overview

This runbook provides diagnostic and remediation steps for authentication failures, expired tokens, OIDC audience/issuer mismatches, AppKey scope rejection, and downstream secret provider errors in Model Context Gateway (MCG).

2. Common Authentication Failure Scenarios

Failure Scenario	Error Code / Symptom	Primary Cause
OIDC Audience Mismatch	`401 Unauthorized: SecurityException: JWT audience 'api://wrong' does not match configured issuer/audience`	IdP token issued for wrong audience or scope
OIDC Issuer Mismatch	`SecurityException: Token issuer 'https://idp.old.com' does not match expected 'https://idp.new.com'`	Issuer mismatch per RFC 9207 security rules
AppKey Expiration / Revocation	`401 Unauthorized: AppKey expired on 2026-08-01` or `Key revoked`	AppKey expiration date reached or key explicitly revoked
Insufficient AppKey Scope	`403 Forbidden: AppKey scope 'server:docker' does not grant access to 'server:homeassistant'`	AppKey lacks target server, category, or tool scope
Expired Vault / Secret Provider	`401 Unauthorized: Downstream backend returned 401 Unauthorized`	HashiCorp Vault token expired or downstream API key rotated

3. Diagnostic Procedures & Remediation

Scenario 1: OIDC Audience and Issuer Mismatch

Inspect Inbound JWT Claims: Decode the Bearer token using jwt.ms or jq to verify iss (issuer) and aud (audience).

Verify Gateway Configuration: Ensure Oidc:Authority matches iss exactly (including trailing slashes) and Oidc:Audience matches aud.

json

{
  "Oidc": {
    "Authority": "https://authentik.corp.internal/application/o/mcg/",
    "Audience": "mcg-gateway-client-id"
  }
}

Issuer Binding Validation: MCG strictly validates RFC 9207 issuer parameter (iss) on token exchange and proxy requests. Mismatched issuers trigger SecurityException and log an audit event (sp_InsertAuditLog).

Scenario 2: AppKey Scope & Access Rejection

When an AppKey produces 403 Forbidden:

Check AppKey prefix and scope definition using Admin tool manage_appkeys(action: "get_limits", prefix: "mcp-usr-...") or Web UI (AppKeys tab).
Ensure the key possesses one of the following required scopes:
- all or *: Unrestricted access across all backend servers.
- server:<targetServerId>: Explicit grant for target server.
- category:<categoryName>: Grant for all servers in category.
- tool:<toolName>: Grant for explicit tool.

Scenario 3: Secret Provider & Vault Token Expiry

When downstream backend connections fail with 401 Unauthorized:

Test Vault connectivity using Admin tool manage_providers(action: "test_vault").
Verify HashiCorp Vault AppRole token TTL and renewal settings. CompositeSecretRetriever automatically attempts token renewal before expiration.
If using AesMasterKey / local key file (./data/.master.key), ensure the file exists and permissions allow gateway read access.

4. Audit Log Querying for Auth Events

Query historical authentication failures via Admin MCP Server:

json

{
  "name": "manage_system",
  "arguments": {
    "action": "query_audit",
    "category": "security",
    "limit": 50
  }
}

Troubleshooting Authentication & Token Failures ​

1. Overview ​

2. Common Authentication Failure Scenarios ​

3. Diagnostic Procedures & Remediation ​

Scenario 1: OIDC Audience and Issuer Mismatch ​

Scenario 2: AppKey Scope & Access Rejection ​

Scenario 3: Secret Provider & Vault Token Expiry ​

4. Audit Log Querying for Auth Events ​

5. Related Operations Guides ​