# Troubleshooting > Common symptoms when running agents on runstate, what usually causes them, and where to look. Source: https://docs.getrunstate.com/troubleshooting/ Start with the error `code` (every SDK error has one) and the [error table](https://docs.getrunstate.com/errors/). For a stuck run, `rs.diagnostics.run(runId)` and the console's **Resources** and **Tasks** pages show who holds what and what is waiting. ## Connecting ### Calls fail with `UnavailableError` after about 30 seconds The SDK can't reach the API. Most often `RUNSTATE_BASE_URL` isn't set, so the client is trying `http://localhost:8080`. Set it to `https://api.getrunstate.com`. Each attempt times out after `requestTimeoutMs` (10 s) and is retried twice, which is where the 30 seconds come from. ### `ConfigError: missing apiKey` or `missing spaceId` `RUNSTATE_API_KEY` or `RUNSTATE_SPACE_ID` isn't set in the environment of the process that constructs the client, and wasn't passed as an option. ### `FORBIDDEN` on some calls but not others The key lacks a permission; the error message names it (`credential lacks resource_config`). Creating queues, pools, quotas, budgets, barriers, work limits or webhook destinations, and changing space limits, need `resource_config`. Diagnostics, usage and limits reads need `usage_read`. Runtime calls (runs, claims, messages, tasks, taking quota and pool units, reserving budget) need `coordination_write`. See [Configuration & auth](https://docs.getrunstate.com/configuration/#api-keys-and-permissions). `FORBIDDEN` without that message means the key isn't allowed for this space. ### `RATE_LIMITED` with only a few workers A space allows its plan's requests per minute (600 on Free) unless its `requestsPerMinute` was set lower. Current SDKs long-poll in `consume()` (`waitMs`, default 20 s), so an idle worker makes about three requests a minute and its empty waits, like lease renewals, don't count toward the rate. SDK versions before 0.2.0 poll every `pollMs` (250 ms), up to 240 requests a minute per idle worker, and those empty polls do count: upgrade the SDK, or raise `pollMs`. If you call `receive()` or `admit()` in your own loop, pass `waitMs` / `wait_ms` instead of sleeping between calls ([request rate](https://docs.getrunstate.com/limits/#request-rate)). ## Workers don't get work ### The worker runs but never receives tasks - **Different run.** Workers only receive tasks submitted under the exact run they attach to (not its parent or children). Check that `rs.scope(runId)` uses the id the producer submitted to. - **Task group members** are delivered on the group's own run: use `(await group.status()).scopeId`. - **Different queue name.** Queue names are per space and case-sensitive. - **Admission.** Tasks submitted with `requirements` or `workLimit` are still visible to `receive()`/`consume()`, which bypass admission. If you use admission, workers should call `admit()`. ### `NotFoundError: mailbox "work" not found` The queue doesn't exist in this space yet. Call `rs.mailboxes.ensure(name)` before submitting, consuming or creating a task group on it. The same applies to pools, quotas, budgets and work limits. ### A task stays `PENDING` - No worker is consuming that queue under that run. - With admission: a required pool or quota is short, the work limit is full, or a quota has a cooldown. `admit()` returns `null` until everything is available; check **Resources** in the console. - The run was cancelled or its deadline passed. `rs.scope(runId).status()` shows `effective: 'INACTIVE'`. ## Leases and ownership ### `LeaseLostError`, or a lease's `signal` aborts, during long work The SDK couldn't renew in time, so it gave the lease up. Common causes: - **The event loop was blocked.** Renewal runs on timers in your process. CPU-heavy synchronous work in Node.js, or blocking calls inside an `async def` in Python, stop renewals. Move heavy work off the event loop (worker threads, `asyncio.to_thread`) or use a longer `leaseSeconds`. - **The lease is too short** for your network or pauses. Try 60 to 120 seconds for long model calls. - **The run was cancelled.** Renewals are refused after cancellation, so every held lease in the run is lost on its next renewal. That is the intended stop signal. ### `CLAIM_HELD` but nobody is working on the key The previous owner probably crashed and its lease hasn't expired yet; it will within `leaseSeconds` of its last renewal. `rs.diagnostics.run(runId)` lists holders and their earliest lease expiry, and `expiredRecent` shows leases that just expired. Use `wait: true` to queue behind the owner instead of failing. ### A pool stays full after agents finish - Leases from crashed agents return only when they expire. - **Pool units taken through `admit()` return when the delivery is settled** (`complete()`, `retry()`, `reject()`) or its lease runs out. If units stay held, a worker is holding an unsettled delivery, or a cancelled task's delivery is still inside its lease. See [admission](https://docs.getrunstate.com/guides/share-capacity/#things-to-know-about-admission). - The organization may be at its "agents working at once" limit (`CONCURRENCY_LIMITED`); acquisitions wait through it. ## Tasks and results ### The task's `attempt` keeps increasing The work is being retried. Either the handler throws (check `onError` / `on_error`), or the work outlasts its lease: `receive()` and `admit()` don't renew, so a delivery held past `leaseSeconds` is handed to another worker. After 5 attempts the task becomes `FAILED`. ### `IDEMPOTENCY_CONFLICT` when submitting A task with that key already exists in the run with different input. Inputs are compared by content fingerprint, so a timestamp or random id inside the input makes every retry "different". Keep volatile values out of the input, or use a different key. ### `submit()` returns an old result instead of doing the work again Task keys are permanent within a run: submitting an existing key with the same input returns the existing task, even if it finished long ago. Use a new key or a new run to redo work. ### `result()` throws `WaitTimeoutError` Only your wait ended. The task is still running on the server. Call `result()` again later, or re-attach with `rs.scope(runId).task(taskId)`. ### A worker keeps polling after the run's deadline After a deadline passes, requests fail with `DEADLINE_EXCEEDED`, which `consume()` treats as a temporary error and keeps polling through. Stop the worker with its `signal` / `stop_event` when your run ends, or cancel the run (cancellation stops `consume()`). ## Capacity and completion ### `quota.take()` waits much longer than the window - A cooldown is active: check `cooldownUntil` in `quota.status()`. - The queue is strict FIFO, so a waiter asking for many units holds back smaller requests behind it. - Waiters left by crashed processes stay queued until their time-to-live and can be granted units first. ### `BACKLOG_FULL` when sending or submitting The space has more queued and in-progress messages than its `maxBacklog` (1,000 by default). Add workers, or raise `maxBacklog` ([per-space limits](https://docs.getrunstate.com/limits/#per-space-limits)). ### An `N_ACCEPTED` group never finalizes A group stays open until you call `group.close()`, `expectedMembers` members have joined, or its deadline passes. Only a closed group can fail with `threshold_unreachable`, so a group that is still open and has too few members that can succeed waits. Call `close()` once you've added the members, or create the group with `expectedMembers` or a `deadline`. See [close the group](https://docs.getrunstate.com/guides/completion-and-cancellation/#close-the-group). ## Python ### `RuntimeWarning: coroutine 'Delivery.complete' was never awaited` The handler is a plain `def`, but delivery methods are coroutines. Make the handler `async def` and `await delivery.complete(...)`. This applies to the blocking `Runstate` client too. ### The blocking `Runstate` client hangs inside a handler An `async def` handler runs on the blocking client's own event loop, so calling a blocking method from it waits on itself. Inside `async def` handlers, `await` the async objects you were given; call blocking methods only from plain `def` handlers or outside handlers. See [the blocking client](https://docs.getrunstate.com/sdk/python/#blocking-client). ## Webhooks ### Signatures don't verify Compute the HMAC over the **raw** request body exactly as received, prefixed by the `x-runstate-timestamp` value and a dot, using the signing secret from destination creation. Re-serializing parsed JSON changes the bytes. See [Webhooks](https://docs.getrunstate.com/guides/observability/#webhooks).