feat: Enhance press-agent communication

[20vikash]

• Enhanced auth: Agent now uses a long-lived HS256 token that Press can verify for authenticated agent callbacks.
• Token regeneration: Tokens can be regenerated automatically without agent downtime.
• Poll -> Push architecture: Press no longer polls agents for job updates. Agents now push job updates to Press in real time.
• Retry support: Agent retries failed job update deliveries when Press is temporarily unreachable.
• Undelivered jobs recovery: Agents poll press every 10 seconds to run undelivered jobs.

Related PR
Agent

[greptile-apps]

Greptile Summary

This PR introduces a significant architectural overhaul of Press–Agent communication: polling is replaced with agent-pushed updates, and shared-secret auth is replaced with per-server Ed25519 tokens stored in a new AgentAuth doctype. Many issues identified in earlier review rounds have been addressed (server filter on update_job, enqueued processing, reload-inside-lock for rotation, DoesNotExistError guard, and correct Ansible status checks).

• New auth layer: agents present a signed X-Agent-Token header verified via Ed25519 public keys cached in Redis; dual-key verification supports zero-downtime token rotation with a 600 s overlap window.
• Push endpoints: new update_job endpoint with server-scoped job lookup; retry_poll scheduler reconciles any undelivered updates every minute using a Redis set.
• Token lifecycle: a daily scheduler pre-rotates tokens within 7 days of expiry using AgentAuth._regenerate_token, running an Ansible playbook under a distributed lock.

Confidence Score: 4/5

The core auth and push-update paths are functional, but a handful of edge cases in the rotation flow and realtime update logic warrant a closer look before merging.

The rotation mechanism leaves regenerate_public_key populated in the database after a successful rotation, relying solely on live agent traffic to clear it. On a dormant server this could block future automated rotations indefinitely. The exp claim is computed from a timezone-stripped naive datetime, which will be wrong on non-UTC hosts. The sadd for undelivered jobs fires unconditionally regardless of the feature flag, building up a backlog silently in poll mode.

press/press/doctype/agent_auth/agent_auth.py (rotation cleanup), press/press/doctype/server/server.py (timestamp computation), press/press/doctype/agent_job/agent_job.py (unconditional sadd and per-update DB query)

Important Files Changed

Filename	Overview
press/agent.py	Adds Ed25519 token verification methods; length check, DoesNotExistError guard, and server-identity claim are correctly implemented. Expiry check retains a 60 s post-expiry grace window (flagged in previous review).
press/api/agent_auth.py	Thin helper that extracts X-Agent-Token, instantiates Agent, and delegates to extract_and_verify_token; logic is straightforward and correct.
press/api/callbacks.py	New update_job endpoint correctly enqueues processing, adds the server filter to prevent cross-server job manipulation, and checks for missing job docs; all issues from prior review rounds appear addressed.
press/press/doctype/agent_auth/agent_auth.py	Key rotation logic is mostly correct after previous fixes (reload inside lock, 600 s TTL cache); however regenerate_public_key is never cleared by the rotation flow itself, creating an edge-case where future rotations may be silently skipped on dormant servers.
press/press/doctype/server/server.py	Key generation, signing, and initial setup look correct; _setup_agent_auth early-return guard and proper auth.save() after Ansible success address prior concerns. sign_agent_token uses a timezone-stripped naive datetime for timestamp(), which gives a wrong exp claim on non-UTC hosts.
press/press/doctype/agent_job/agent_job.py	Undelivered-jobs retry via Redis set is well-structured; srem is correctly placed in the else clause. Two new concerns: unconditional sadd regardless of push_feature, and a per-update DB query for all step docnames in publish_update.
press/hooks.py	Correctly registers the daily regenerate_token scheduler and the per-minute retry_poll scheduler.

Prompt To Fix All With AI

Fix the following 4 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 4
press/press/doctype/agent_auth/agent_auth.py:42-43
**`regenerate_public_key` not cleared after successful rotation**

`_regenerate_token` sets `regenerate_public_key` in the DB and relies on `get_regenerate_public_key()` (called on every agent request) to clear it once the Redis cache expires after 600 s. The rotation flow itself never clears the field. If a server goes quiet for more than 600 s after the cache TTL expires — and then the daily scheduler fires the next pre-expiry rotation — `self.reload()` on line 39 will find `regenerate_public_key` still populated and return early, silently skipping the rotation. A dormant-but-still-registered server could end up with an expired token and no automatic way to recover. Adding a DB clear of `regenerate_public_key` at the end of a successful rotation (or inside `_setup_agent_auth` on success) would close this gap.

### Issue 2 of 4
press/press/doctype/agent_job/agent_job.py:197
**`sadd` called unconditionally regardless of the `push_feature` flag**

`frappe.cache().sadd("undelivered_jobs", ...)` fires on every callback delivery failure, even when `push_feature` is disabled and `retry_poll` is a no-op. In poll-only deployments, the set grows without bound (one entry per unique server with any failure), and when `push_feature` is eventually enabled, `retry_poll` will immediately process the entire accumulated backlog in a single scheduler tick. Guard the `sadd` with the same flag check used in `retry_poll` to avoid this.

### Issue 3 of 4
press/press/doctype/agent_job/agent_job.py:458-467
**Extra DB query per `publish_update` call scales with step count**

`frappe.get_all("Agent Job Step", ...)` is now executed on every `publish_update` invocation. `publish_update` is called from `process_job_updates` on each polled or pushed status change, so a job with N steps triggers N+1 additional socket publishes and one extra DB round-trip on every update cycle. For jobs with dozens of steps updating at high frequency this adds measurable overhead. Consider caching the step names when the job is first processed, or limit this realtime push to a single `list_update` event that the client can use to re-fetch rather than pushing per-step `doc_update` events.

### Issue 4 of 4
press/press/doctype/server/server.py:1938-1945
**`expires_in.timestamp()` on a timezone-stripped naive datetime**

`datetime.datetime.now(datetime.timezone.utc)` yields a UTC-aware datetime. After `.replace(tzinfo=None)` it becomes a naive datetime. Calling `.timestamp()` on a naive datetime interprets it as local time, so on a server in a non-UTC timezone the `exp` claim in the JWT will be offset by the UTC delta — the token effectively expires earlier or later than intended. Remove the `.replace(tzinfo=None)` from the `expires_in` assignment so the aware datetime is converted correctly; strip the timezone only when writing to the Frappe `Datetime` field.

```suggestion
		expires_in_aware = datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(days=90)

		# Strip tzinfo only for the Frappe Datetime field (which is stored as naive UTC)
		expires_in = expires_in_aware.replace(tzinfo=None)

		payload = {
			"server": self.name,
			"exp": int(expires_in_aware.timestamp()),  # 3 month
		}
```

_{Reviews (9): Last reviewed commit: "fix(agent): Throw permission error if ve..." | Re-trigger Greptile}