Deployment Monitoring

Deployment Monitoring links recorded implementation commits to CI/CD webhooks from Vercel, Netlify, Railway, Render, or self-hosted Coolify. If a deployment fails, DevSpec classifies the error, records whether the deployment operation can be retried, and attaches code-fix evidence to a tracked item. It never routes that item to an agent.

How It Works

The deployment monitoring pipeline follows a webhook-driven flow: your CI/CD platform sends a webhook when a deployment finishes. DevSpec receives the event, fetches build logs, classifies the failure with Claude, and decides whether to retry or escalate.

• Your CI/CD platform (Vercel, Netlify, Railway, Render, or Coolify) sends a deployment webhook to DevSpec.
• DevSpec records the event and links it to the originating action item and deployment target.
• If the deployment succeeded, DevSpec records the success and notifies relevant parties.
• If the deployment failed, DevSpec fetches the build logs via the platform API.
• Claude classifies the failure into an error category (code, config, dependency, infra, test, unknown) and provides a human-readable reason.
• The retry gate decides whether the deployment operation itself can be retried. A failure requiring code changes becomes a tracked fix request; it is not routed to an agent.
• If the deployment itself is retryable and under the max retry limit, DevSpec may retry that deployment operation according to target policy.
• If code changes are required, DevSpec records a fix action item with enriched error context. A user must explicitly request an exact authorized agent to implement it; the agent then calls reserve_work_items and claim_work_item.
• If not retryable or retries are exhausted, the deployment failure is escalated with a notification.

1. Setting Up Vercel

Connect Vercel to DevSpec so deployment events are forwarded automatically.

Step 1 — Create a deployment target in DevSpec

In your DevSpec project, go to Settings > Deployment Targets and click "Add Deployment Target". Choose "Vercel" as the platform and give it a name (e.g. "Production"). Save it — DevSpec will display a webhook URL. Copy the URL.

Step 2 — Open the Vercel webhook settings

Navigate to your Vercel dashboard and go to Settings > Webhooks. Click "Add Webhook".

Step 3 — Choose projects

Under "Projects", select "Specific Projects" and choose only the Vercel project you want to monitor.

Step 4 — Select events

Under "Deployment Events", check the following: "Deployment Succeeded" (deployment.succeeded), "Deployment Error" (deployment.error), and "Deployment Canceled" (deployment.canceled). Optionally check "Deployment Created" (deployment.created) for real-time in-progress tracking. Leave all other event categories unchecked.

Step 5 — Paste the endpoint URL

Paste the webhook URL you copied from DevSpec into the "Endpoint" field at the bottom. It will look like this:

https://devspec.ai/api/webhooks/deployment/{your-project-id}/{your-target-id}

Step 6 — Save and copy the signing secret

Click "Create Webhook". Vercel will show a confirmation page with a signing secret — copy it immediately, as it will not be shown again. This secret is used to verify that incoming webhooks are authentically from Vercel.

Step 7 — Save the secret in DevSpec

Go back to your deployment target in DevSpec, click Edit, and paste the Vercel signing secret into the "Webhook Secret" field. Optionally also add your Vercel API token — this lets DevSpec fetch build logs when a deployment fails.

Step 8 — Verify the connection

Trigger a test deployment in Vercel. You should see a deployment event appear in DevSpec within a few seconds.

2. Setting Up Netlify

Connect Netlify to DevSpec for deployment monitoring.

Step 1 — Open Netlify site settings

Go to your Netlify dashboard, select the site, and navigate to Site Configuration > Notifications.

Step 2 — Add an outgoing webhook

Under "Outgoing webhooks", click "Add notification" and select "Outgoing webhook".

Step 3 — Configure the webhook URL

Enter the DevSpec webhook URL:

https://devspec.ai/api/webhooks/deployment/{your-project-id}/netlify

Step 4 — Select the deploy events

Add webhooks for "Deploy succeeded" and "Deploy failed" events.

Step 5 — Add your Netlify API token in DevSpec

In the deployment target settings, paste your Netlify personal access token so DevSpec can fetch build logs on failure.

3. Setting Up Railway

Connect Railway to DevSpec for deployment monitoring.

Step 1 — Open Railway project settings

Go to your Railway dashboard, select the project, and navigate to Settings > Webhooks.

Step 2 — Add the DevSpec webhook URL

Enter the DevSpec webhook URL:

https://devspec.ai/api/webhooks/deployment/{your-project-id}/railway

Step 3 — Select deployment events

Subscribe to "Deployment Success" and "Deployment Failed" events.

Step 4 — Add your Railway API token in DevSpec

In the deployment target settings, paste your Railway API token to enable build log fetching.

4. Setting Up Render

Connect Render to DevSpec for deployment monitoring.

Step 1 — Open Render service settings

Go to your Render dashboard, select the service, and navigate to Settings > Notifications.

Step 2 — Add a webhook notification

Click "Add Notification" and choose "Webhook".

Step 3 — Configure the webhook URL

Enter the DevSpec webhook URL:

https://devspec.ai/api/webhooks/deployment/{your-project-id}/render

Step 4 — Select events

Subscribe to "Deploy succeeded" and "Deploy failed" events.

Step 5 — Add your Render API key in DevSpec

In the deployment target settings, paste your Render API key to enable log fetching for failed deployments.

5. Setting Up Coolify

Coolify is self-hosted, so it's tracked a little differently. You choose how much to connect — from a webhook with no inbound access at all, up to an API token for exact commits and build logs. The trade-offs are covered at the end of this section.

Coolify's deploy notification is team-wide: one webhook fires for every app in the team. DevSpec handles this with a single project-level URL and routes each notification to the right target by Coolify Application UUID — so you add one DevSpec target per Coolify app you want to track.

Step 1 — Create a deployment target per app

Go to Settings > Deployment Targets > Add Deployment Target. Choose "Coolify", give it a name, then set:

• Coolify Application UUID — the app this target tracks. Open the app in Coolify and copy the last segment of the dashboard URL (…/application/<uuid>) — a short random ID, not your app's public domain. (It's also the uuid= value in the app's Deploy Webhook URL.)
• Branch Filter — the branch this app deploys (e.g. main or staging); used to resolve the commit from your connected repository's data.

You don't set a webhook secret — Coolify's webhook is unsigned, so DevSpec generates a strong one for you and builds it into the URL below. (You can rotate it any time by editing the target → Regenerate webhook secret, which gives you a fresh URL to paste back into Coolify.)

Repeat for each Coolify app you want to track. Copy the webhook URL — it's the same project-level URL for all your Coolify targets.

Step 2 — Add ONE Webhook notification in Coolify

In Coolify, go to Settings → Notifications → Webhook (configured per team) and enable it. When you save the target, DevSpec shows the complete webhook URL — copy it and paste it into Coolify as-is. It already includes the generated webhook secret as a ?secret= token (the secret is encrypted at rest afterward, so grab the URL while it's shown):

https://devspec.ai/api/webhooks/deployment/{your-project-id}/coolify?secret=<your-secret>

Enable the Deployment Success and Deployment Failure events. DevSpec routes each app's deploys to the matching target automatically; deploys for apps you haven't added as targets are ignored.

That's the zero-exposure baseline — no API token, no inbound access. DevSpec records every success and failure; because Coolify's webhook doesn't include the commit, DevSpec resolves it from your connected repository (the branch's HEAD at deploy time).

Optional — connect the Coolify API. Enable Coolify's API (Settings → API), then edit the target and add:

• A read API token + an Instance Base URL (e.g. https://coolify.example.com) → DevSpec fetches the exact commit SHA instead of inferring it. A read token can read your team's resource metadata (apps, config, commits) but not secrets — low risk, and the most accurate option if your Coolify API is reachable from DevSpec.
• A read:sensitive token additionally fetches build logs, enabling AI failure classification and auto-retry. Note that read:sensitive also grants access to your environment variables, secrets, and logs — only use it if you want auto-retry and accept that trade-off.

Least privilege: a Coolify API token can read everything in the team it was created under. If you connect a token, consider putting the app(s) in a dedicated Coolify team and creating the token there, so DevSpec can only ever see that team — and set an expiration so it auto-retires.

6. Backfilling Past Deployments

Webhooks are forward-only — they tell DevSpec about deploys that happen after you connect. To pull in your history as well, use Backfill.

When you connect (or edit) a target that has everything backfill needs, DevSpec kicks off a backfill automatically in the background — so you usually don't have to do anything. You can also run it on demand: on Settings → Deployment Targets, eligible targets show a history icon (↻); click it to import recent deployments from the provider's API.

Backfill needs an API token on the target plus the provider's scoping config:

• Coolify — the Instance Base URL + Application UUID (exactly what powers exact-commit tracking).
• Vercel — the Vercel Project ID (Settings → General → Project ID on the Vercel project), so it imports only that project's deployments.

Webhook-only targets (no token) can't backfill.

What to expect:

• It imports the most recent deployments (the latest ~50), newest first, stamped with their real deploy time so they land correctly on the timeline.
• It's deduplicated — anything DevSpec already has (from a live webhook or an earlier backfill) is left untouched, so it's safe to run more than once. The toast tells you how many were imported vs. already present.
• Imported deploys are linked to action items the same way live ones are (commit tag, branch, or commit reference).
• Backfill is silent: historical deploys never fire notifications, retries, escalations, or health changes — it only fills in the record. Deploys with no VCS commit (e.g. docker-image builds) are skipped.

Understanding Results

Deployment events are displayed on the action item detail page with status badges and classification details.

Status badges

• Green "succeeded" badge: the deployment completed successfully.
• Red "deployment_failed" badge: the deployment failed.

Error categories

• code — a bug in the pushed code. Record a fix item; a user explicitly requests its implementation.
• dependency — a missing or incompatible dependency. Retry the deployment only when the target operation can genuinely resolve it; otherwise record a fix item.
• config — environment variable or configuration issue. Escalated.
• infra — platform outage or infrastructure problem. Escalated.
• test — test suite failure. Record evidence and request a named fix; do not auto-route work.
• unknown — could not be classified. Escalated.

Retry behavior

Each deployment target has a configurable max retry count (default 2). A deployment retry re-runs the deployment on your platform and nothing else — it never routes work to an agent. When platform retry is safe and under the limit, DevSpec retries the deployment. When code changes are required, DevSpec records a fix action item with the error context; a user explicitly requests an exact authorized agent, which calls reserve_work_items then claim_work_item. Exhausted deployment retries are escalated by notification.

Troubleshooting

Common issues and how to resolve them.

Webhook events not arriving in DevSpec

Your CI/CD platform is sending webhooks but DevSpec does not show any deployment events.

• Verify the webhook URL contains your correct DevSpec project ID.
• Check the webhook delivery logs in your CI/CD platform for HTTP errors.
• Ensure your DevSpec instance is publicly accessible (not behind a firewall).
• Confirm you selected the correct deployment events (succeeded and failed).

Build logs showing as "unavailable"

Deployment failures are recorded but the build log excerpt says "Logs unavailable".

• Check that an API token is configured for the deployment target in DevSpec.
• Verify the token has permission to read deployment logs on the CI/CD platform.
• Some platforms rate-limit log API calls — wait a few minutes and check again.

Classification seems incorrect

Claude classified the error in the wrong category.

The classification uses Claude Sonnet to analyze build logs. Ambiguous errors (e.g. a missing env var that looks like a code error) can be misclassified. The full build log excerpt is available in the deployment event card for manual review. If the classification is wrong, the escalation path ensures a human reviews it.

Action item stuck in "deployment_failed" status

The action item shows deployment_failed but you want to retry manually.

• Keep or create a fix action item with the deployment evidence, then explicitly request an exact authorized agent to implement it. The agent reserves and claims that named item; changing status alone does not deliver work.
• Alternatively, fix the issue manually and push a new deployment — DevSpec will ingest the new webhook event.

Multiple deployment targets for one project

You have staging and production environments and want monitoring for both.

Each deployment target is configured independently with its own webhook URL, API token, and retry settings. Add a separate deployment target in DevSpec project settings for each environment. The webhook URL includes the target ID to route events correctly.

Webhook signature verification failing

The CI/CD platform reports successful delivery but DevSpec rejects the webhook.

• Check that the webhook secret configured in DevSpec matches the one in your CI/CD platform.
• Some platforms send a test event on creation — verify these are being accepted.
• Check server logs for signature mismatch errors.

Notifications not sent on escalation

A deployment was escalated but no notification was received.

• Verify the action item has an assigned user — notifications are sent to the assigned developer.
• Check notification preferences in user settings.
• Look at the Inngest dashboard for failed notification jobs.