Skip to content

Workflow World returns 404 not-found for runs created by newly-deployed functions (@workflow/world-vercel) #2735

Description

@olgen

Summary

Since ~2026-07-01 08:57 UTC, workflow runs created by any newly-created deployment fail: writes to the managed Workflow World (vercel-workflow.com) return HTTP 404 not-found for a run the deployment just created. Runs created by an older deployment of the same project keep working, and rolling the production alias back to a pre-08:57-UTC deployment fully restored service. The application code and dependency lockfile are byte-identical across the working and failing deployments, so this looks like a platform-side Workflow World registration/provisioning problem tied to newly-created deployments rather than an app change.

A Vercel support case has been filed with the full private details (team/project/deployment IDs): Support ticket 01O1qb7pAz2Gs2tP.

Environment

  • @workflow/core 5.0.0-beta.26
  • @workflow/world-vercel 5.0.0-beta.22
  • workflow 5.0.0-beta.26
  • Runtime: bun1.x, Nitro (preset vercel), compatibility date 2026-07-01
  • Region observed: fra1
  • Deployed via source build (bun install --ignore-scripts on Vercel; lockfile unchanged between working/failing builds)

Primary error (repeats for essentially every new run)

[workflow-sdk] Background flush of step stream ops failed
  run:  wrun_01KWEF3VV779FRP13S88W078ZN
  step: step_01KWEF3VV7E0FPGZESQWQGK8W2
  error: Stream write failed: HTTP 404
  PUT https://vercel-workflow.com/api/v2/runs/wrun_01KWEF3VV779FRP13S88W078ZN/stream/strm_01KWEF3VV779FRP13S88W078ZN_user
  response: {"success":false,"error":"not-found","message":"workflow run wrun_01KWEF3VV779FRP13S88W078ZN not found","meta":{"entityType":"workflow run","entityId":"wrun_01KWEF3VV779FRP13S88W078ZN"}}

The run is being processed by the same deployment's POST /.well-known/workflow/v1/flow handler at the time its stream write is rejected as not-found — i.e. the World does not have a run the deployment itself just started.

Secondary symptom (cascade)

Pre-existing runs also became unresolvable, so delivering to an already-active session throws WorkflowRunNotFoundError, and the fallback new run then cannot claim its hook token:

WorkflowRunNotFoundError: Workflow run "wrun_01K...redacted..." not found
HookConflictError: Hook token "<redacted>" is already in use (run "wrun_01K...redacted...")

Net effect: an inbound event can neither resume the existing run nor start a new one.

Timeline (UTC)

Time (UTC) Event
08:31:05 Last successful run wrun_01KWECV85FHE95SHK6MCSXX4CN
08:57:34 New production deployment created
09:10:18 First failing run wrun_01KWEF321VKHFXQ876FZE8RG2A
~10:04 Fresh deploy from identical code — also fails
~10:08 Rolled production alias back to the 07:43-UTC deployment — service restored

What we ruled out

  • No application code change between the working and failing deployments.
  • Identical dependency pins in the uploaded package.json / bun.lock of the failing deployment (@workflow/core@5.0.0-beta.26, @workflow/world-vercel@5.0.0-beta.22).
  • No project env-var changes (all last modified 9 days prior).
  • Not a serialization/durable-format-version change (format version unchanged).

The only differentiator between working and failing runs is which deployment created them: deployments created before ~08:57 UTC register/resolve runs in the World fine; deployments created after do not.

Expected

A run created by a freshly-deployed function should be registered in the Workflow World so subsequent step/stream writes for that run resolve, regardless of deployment age.

Actual

Runs created by post-08:57-UTC deployments are absent from the World; every step-stream PUT …/runs/<run>/stream/<strm> returns 404 not-found, failing the run.

Questions

  1. Is there a per-deployment Workflow World provisioning/binding step that can fail for a new deployment (leaving @workflow/world-vercel unable to persist runs that deployment creates)?
  2. How can a new deployment be created safely without re-triggering this? (We are currently pinned to a 2-hour-old deployment.)
  3. Is there a supported way to release/expire orphaned hook tokens whose owning runs are no longer resolvable?
  4. Was there a Workflows platform change/incident in fra1 around 2026-07-01 08:57 UTC?

Additional context

Private identifiers (team/project/deployment IDs, affected hook tokens) are in Vercel support ticket 01O1qb7pAz2Gs2tP.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions