fix(webhooks): make the request metadata opt-in per webhook

The four commits below make the generic webhook do what its Setup Instructions
promise. They do it through a provider-level capability, which applies to every
generic webhook row the moment it deploys: each one begins accepting GET, PUT,
PATCH and DELETE, and each one's workflow input gains `method` and `headers`,
on POST deliveries too. No webhook owner chose either.

Gate both behind `providerConfig` flags written by two switches, off by default.
A webhook deployed before these existed has neither flag, so it answers POST
only and its input is exactly the body, as before. `query` stays ungated: it is
dropped today, only appears when the caller's own URL carries it, and yields to
a body field of the same name.

Generalize the Microsoft Teams challenge fix. Every challenge handler runs
before the webhook lookup and matches on payload shape alone, so any of them
will answer a delivery addressed to another provider on the same path. Gate
them centrally to POST via `challengeMethods`, which WhatsApp widens to GET for
Meta's handshake, rather than guarding one handler inline.

Also:

- Widen the credential header denylist to 24 names and withhold the webhook's
  own token by value as well as by name, since a denylist is leaky by
  construction.
- Condition the `method` and `headers` trigger outputs on their switches, so
  the reference dropdown cannot offer a field the webhook will not send.
- Give PUT, PATCH and DELETE their own contracts instead of reusing the POST
  one, whose `method: 'POST'` had become untrue.
- Parse, challenge and generate a request ID once per delivery rather than
  twice on GET, which was logging one request under two IDs.
- Offer the challenge handlers the request before admission, so Meta's GET
  handshake cannot be answered with a 429 by a busy instance.
- Answer every non-POST rejection with the same 405 plus `Allow`, whether the
  path is unknown, holds only non-path triggers, or holds a trigger that has
  not opted in.
- Read flags through a helper treating only `true`/`'true'` as on: the editor
  writes booleans, but a YAML- or Copilot-authored workflow can write the
  string `'false'`, which is truthy.
- Name the methods switch "Accept Other HTTP Methods": HEAD and OPTIONS still
  answer 405, so claiming "all" would reintroduce the overstatement this whole
  change set exists to remove.
- Drop the per-delivery metadata warn logs to debug.

Author: waleedlatif1

apps/sim/app/api/webhooks/trigger/[path]/route.test.ts

@@ -462,6 +462,10 @@ vi.mock('postgres', () => vi.fn().mockReturnValue({}))
 
 process.env.DATABASE_URL = 'postgresql://test:test@localhost:5432/test'
 
+import {
+  handlePreLookupWebhookVerification,
+  handleProviderChallenges,
+} from '@/lib/webhooks/processor'
 import { DELETE, GET, PATCH, POST, PUT } from '@/app/api/webhooks/trigger/[path]/route'
 
 describe('Webhook Trigger API Route', () => {
@@ -683,14 +687,68 @@ describe('Webhook Trigger API Route', () => {
     })
   })
 
+  /**
+   * Both handshakes are answered from the request alone, before any webhook lookup, so their
+   * order relative to each other and to the load-shed gate is the behavior — and it is invisible
+   * to every other test here, which is how an earlier refactor inverted it unnoticed.
+   */
+  describe('pre-lookup handshake ordering', () => {
+    /**
+     * Meta verifies a WhatsApp URL with a GET challenge. Answering it behind the load-shed gate
+     * means a busy instance returns 429 and the webhook silently fails to verify, at setup time
+     * only — so the challenge must be answered without taking a ticket at all.
+     */
+    it('answers a provider challenge without taking an admission ticket', async () => {
+      vi.mocked(handleProviderChallenges).mockResolvedValueOnce(
+        new NextResponse('hub-challenge-123', { status: 200 })
+      )
+
+      const req = createMockRequest(
+        'GET',
+        undefined,
+        {},
+        'http://localhost:3000/api/webhooks/trigger/verify-path?hub.challenge=hub-challenge-123'
+      )
+
+      const response = await GET(req, { params: Promise.resolve({ path: 'verify-path' }) })
+
+      expect(response.status).toBe(200)
+      await expect(response.text()).resolves.toBe('hub-challenge-123')
+      expect(tryAdmitMock).not.toHaveBeenCalled()
+    })
+
+    /**
+     * A challenge is the more specific answer: the provider is echoing a token it chose, where a
+     * pending verification only claims the URL is reachable. Answering the generic 200 first
+     * fails the handshake that actually had a token to return.
+     */
+    it('prefers a provider challenge over a pending setup verification', async () => {
+      vi.mocked(handleProviderChallenges).mockResolvedValueOnce(
+        new NextResponse('hub-challenge-123', { status: 200 })
+      )
+
+      const req = createMockRequest(
+        'GET',
+        undefined,
+        {},
+        'http://localhost:3000/api/webhooks/trigger/verify-path?hub.challenge=hub-challenge-123'
+      )
+
+      const response = await GET(req, { params: Promise.resolve({ path: 'verify-path' }) })
+
+      await expect(response.text()).resolves.toBe('hub-challenge-123')
+      expect(handlePreLookupWebhookVerification).not.toHaveBeenCalled()
+    })
+  })
+
   describe('GET deliveries', () => {
     it('dispatches a GET delivery to a generic webhook', async () => {
       testData.webhooks.push({
         id: 'generic-webhook-id',
         provider: 'generic',
         path: 'get-path',
         isActive: true,
-        providerConfig: { requireAuth: false },
+        providerConfig: { requireAuth: false, acceptOtherMethods: true },
         workflowId: 'test-workflow-id',
       })
 
@@ -707,6 +765,61 @@ describe('Webhook Trigger API Route', () => {
       expect(dispatchResolvedWebhookTargetMock).toHaveBeenCalledOnce()
     })
 
+    /**
+     * The compatibility guarantee for the route: a generic webhook deployed before the flag
+     * existed has no flag, so it answers exactly as it did before — 405, no execution.
+     */
+    it('rejects a GET delivery to a generic webhook that has not opted in', async () => {
+      testData.webhooks.push({
+        id: 'generic-webhook-id',
+        provider: 'generic',
+        path: 'opt-out-path',
+        isActive: true,
+        providerConfig: { requireAuth: false },
+        workflowId: 'test-workflow-id',
+      })
+
+      const req = createMockRequest(
+        'GET',
+        undefined,
+        {},
+        'http://localhost:3000/api/webhooks/trigger/opt-out-path?srcId=123'
+      )
+
+      const response = await GET(req, { params: Promise.resolve({ path: 'opt-out-path' }) })
+
+      expect(response.status).toBe(405)
+      expect(response.headers.get('Allow')).toBe('POST')
+      expect(dispatchResolvedWebhookTargetMock).not.toHaveBeenCalled()
+    })
+
+    /**
+     * Next derives HEAD from the exported GET, so a HEAD probe reaches the same handler. It must
+     * not execute a workflow: scanners and prefetchers send HEAD unprompted.
+     */
+    it('rejects a HEAD probe to a webhook that accepts every declared method', async () => {
+      testData.webhooks.push({
+        id: 'generic-webhook-id',
+        provider: 'generic',
+        path: 'head-path',
+        isActive: true,
+        providerConfig: { requireAuth: false, acceptOtherMethods: true },
+        workflowId: 'test-workflow-id',
+      })
+
+      const req = createMockRequest(
+        'HEAD',
+        undefined,
+        {},
+        'http://localhost:3000/api/webhooks/trigger/head-path'
+      )
+
+      const response = await GET(req, { params: Promise.resolve({ path: 'head-path' }) })
+
+      expect(response.status).toBe(405)
+      expect(dispatchResolvedWebhookTargetMock).not.toHaveBeenCalled()
+    })
+
     it('rejects a GET delivery to a provider that only accepts POST', async () => {
       testData.webhooks.push({
         id: 'stripe-webhook-id',
@@ -742,7 +855,7 @@ describe('Webhook Trigger API Route', () => {
           provider: 'generic',
           path: 'any-method-path',
           isActive: true,
-          providerConfig: { requireAuth: false },
+          providerConfig: { requireAuth: false, acceptOtherMethods: true },
           workflowId: 'test-workflow-id',
         })
 
@@ -762,6 +875,29 @@ describe('Webhook Trigger API Route', () => {
       }
     )
 
+    it('rejects a PUT delivery to a generic webhook that has not opted in', async () => {
+      testData.webhooks.push({
+        id: 'generic-webhook-id',
+        provider: 'generic',
+        path: 'opt-out-path',
+        isActive: true,
+        providerConfig: { requireAuth: false },
+        workflowId: 'test-workflow-id',
+      })
+
+      const req = createMockRequest(
+        'PUT',
+        { event: 'test' },
+        {},
+        'http://localhost:3000/api/webhooks/trigger/opt-out-path'
+      )
+
+      const response = await PUT(req, { params: Promise.resolve({ path: 'opt-out-path' }) })
+
+      expect(response.status).toBe(405)
+      expect(dispatchResolvedWebhookTargetMock).not.toHaveBeenCalled()
+    })
+
     it('rejects a PUT delivery to a provider that only accepts POST', async () => {
       testData.webhooks.push({
         id: 'stripe-webhook-id',
@@ -785,6 +921,35 @@ describe('Webhook Trigger API Route', () => {
       expect(dispatchResolvedWebhookTargetMock).not.toHaveBeenCalled()
     })
 
+    /**
+     * Every non-POST rejection is the same 405, whether the path is unknown, holds only
+     * non-path triggers, or holds a trigger that has not opted in — so a probe cannot tell
+     * a configured path from an unused one.
+     */
+    it('returns the same 405 for a DELETE to a non-path trigger as to an unknown path', async () => {
+      testData.webhooks.push({
+        id: 'internal-webhook-id',
+        provider: 'sim',
+        path: 'internal-path',
+        isActive: true,
+        providerConfig: {},
+        workflowId: 'test-workflow-id',
+      })
+
+      const req = createMockRequest(
+        'DELETE',
+        undefined,
+        {},
+        'http://localhost:3000/api/webhooks/trigger/internal-path'
+      )
+
+      const response = await DELETE(req, { params: Promise.resolve({ path: 'internal-path' }) })
+
+      expect(response.status).toBe(405)
+      expect(response.headers.get('Allow')).toBe('POST')
+      expect(dispatchResolvedWebhookTargetMock).not.toHaveBeenCalled()
+    })
+
     it('returns 405 for a DELETE to an unknown path', async () => {
       const req = createMockRequest(
         'DELETE',