Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,16 @@ All notable user-facing changes to this project will be documented in this file.

## Unreleased

- Node version is no longer edited by hand in the portal — it is detected automatically from your running node and shown read-only on your profile, so it always reflects reality instead of whatever was last typed in

- Node upgrades are now detected automatically from live node metrics instead of relying on validators to edit their profile: when the network's first validator adopts a new stable release, that version becomes the official upgrade target, each validator's running version is kept in sync from what their node actually reports, and the node-upgrade points (more for upgrading sooner) are awarded automatically once a validator is seen on the target version — no manual submission or steward review needed

- The Wall of Shame now shows how many consecutive days each validator has gone without being shamed — a clean-uptime streak per node and per network (Asimov and Bradbury), alongside the reasons a streak was broken. An operator's network streak stays alive as long as at least one of their nodes was healthy that day

- The portal now keeps a daily history of each validator's observability: every Grafana sync records whether each node was reporting metrics, reporting logs, and running an up-to-date version, and rolls it up per day (a day counts as shamed if the node was shamed at any point that day). This history is the foundation for upcoming uptime-streak and days-in-shame reporting; it starts accumulating from deploy since past days were never recorded

- The grace period before a validator running an outdated node version is flagged on the Wall of Shame is now configurable via a setting (default three days), instead of a fixed three-day window baked into the code

- Grafana dashboards can now read a dedicated minimal validator roster endpoint that lists every validator wallet with its network, on-chain node address, display name, status, operator address, and (for visible operators) linked account, kept intentionally small and separate from the Wall of Shame so monitoring can join validator identity onto live node metrics (2aaf68f)

- Contribution types can now require evidence URLs in groups, so a single submission can be made to provide both a contract code link (GenLayer Studio import or GitHub repository) and a deployed-contract explorer link (Asimov, Bradbury, or Studio explorer) together; GenLayer explorer contract addresses are now recognized as their own evidence type (25f90869)
Expand Down
34 changes: 26 additions & 8 deletions backend/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,13 @@ backend/
- Validator model with node_version field (OneToOne with User)
- Custom UserManager for email-based auth
- **Views**: `users/views.py`
- `/api/v1/users/me/` - GET/PATCH current user profile (name and node_version editable)
- `/api/v1/users/me/` - GET/PATCH current user profile (name/description/website/socials editable; node_version is NOT editable — Grafana-sourced, display only)
- `/api/v1/users/by-address/{address}/` - Get user by wallet address
- `/api/v1/users/validators/` - Get validator list from blockchain
- **Serializers**: `users/serializers.py`
- UserSerializer - Full user data including validator info
- ValidatorSerializer - Validator node version and target matching
- UserProfileUpdateSerializer - Allows name and node_version updates
- UserProfileUpdateSerializer - Allows name/description/website/socials updates (node_version removed — Grafana is source of truth)
- UserCreateSerializer - Registration

### Authentication
Expand Down Expand Up @@ -101,7 +101,21 @@ backend/

### Node Upgrade (Sub-app)
- **Models**: `contributions/node_upgrade/models.py`
- TargetNodeVersion - Active target version for node upgrades
- TargetNodeVersion - Active target version for node upgrades. Per-network, single
`is_active` per network. The version-shame grace period — how many days after
`target_date` a node still behind is marked version "shame" — is the global
`settings.NODE_VERSION_SHAME_GRACE_DAYS` (default 3, env-overridable), not a per-target field.
- **Version verdict**: `validators/version_status.py::compute_version_status(wallet, target, now, node_version=...)`
is the shared helper (used by the Wall of Shame view and the Grafana sync) that returns
`on`/`warning`/`shame`/`unknown` using the `NODE_VERSION_SHAME_GRACE_DAYS` setting. The viewset's
`ValidatorWalletViewSet._version_context` delegates to it; the Grafana sync passes the
Prometheus-observed version explicitly.
- **Node versions are Grafana-sourced.** Target creation and the `node-upgrade` award are
driven automatically by the Grafana sync (`GrafanaValidatorStatusService._sync_node_versions`
/ `_award_node_upgrade`); the portal no longer lets users edit their node version. The old
`NodeVersionMixin.save()` auto-submission path has been removed — `NodeVersionMixin` now only
holds the version fields + validation + comparison helpers, and `calculate_early_upgrade_bonus`
(reused by the Grafana award). Dedup on the `version {v} [{network}]` notes key is preserved.
- **Admin**: `contributions/node_upgrade/admin.py`
- TargetNodeVersion admin interface
- **Views**: `contributions/views.py`
Expand Down Expand Up @@ -227,17 +241,20 @@ backend/
### Validators
- **Models**: `validators/models.py`
- ValidatorWallet - Synced validator wallet metadata per network. Now also stores Wall of Shame observability state: `metrics_status`, `logs_status` (both `on` / `shame` / `unknown`), and `last_grafana_check_at`.
- ValidatorWalletStatusSnapshot - Daily wallet status snapshots for uptime lookback
- ValidatorWalletStatusSnapshot - Daily wallet rollup. On-chain `status` (owned by the on-chain sync, for uptime lookback) PLUS the latched observability verdict written by the Grafana sync: `metrics_status` / `logs_status` / `version_status`, `metrics_samples` / `logs_samples` counters, and `node_version`. **Metrics and logs latch pessimistically** (worst-of-day: shame at ANY observation → the day is shame). **Version latches optimistically** (best-of-day: a single up-to-date observation → the day is OK, since once a node upgrades that day an earlier stale reading must not shame it; `on` > `warning` > `shame`). A day is "clean" only if `status=='active'` and both sample counters are ≥1 and neither metrics nor logs is `shame` and version is not `shame`. The two syncs write disjoint columns (bulk_create update_conflicts on `(wallet, date)`), so neither clobbers the other.
- ValidatorWalletObservation - Append-only raw log; one row per active wallet per Grafana sync run (`observed_at`, `onchain_status`, `metrics_status`, `logs_status`, `version_status`, `node_version`). Source of truth the daily rollup is materialised from and rebuildable via `rebuild_daily_snapshots`.
- SyncLock - Database-backed sync coordination row with owner token for cross-worker locking
- **Services**: `validators/grafana_service.py`
- GrafanaValidatorStatusService - Polls Grafana Cloud (`/api/ds/query`) Prometheus + Loki datasources and updates `ValidatorWallet.metrics_status` / `logs_status` for `status='active'` wallets, per network. Used by the Wall of Shame cron.
- GrafanaValidatorStatusService - Polls Grafana Cloud (`/api/ds/query`) Prometheus + Loki datasources and updates `ValidatorWallet.metrics_status` / `logs_status` for `status='active'` wallets, per network. The Prometheus query also reads the `version` label from `genlayer_node_info` — **normalised at ingest** in `parse_response` ('v' prefix stripped, capped to the 50-char column; when a node briefly reports two version series right after an upgrade, the higher parseable one wins). Each run writes a `ValidatorWalletObservation` and latches today's `ValidatorWalletStatusSnapshot` rollup (`_record_history`, best-effort — never breaks the live status sync). Observations are retained forever by explicit decision — no pruning in points. Used by the Wall of Shame cron.
- GrafanaValidatorStatusService is also the **source of truth for node versions** (`_sync_node_versions`, best-effort): version detection covers **every reporting node on the network regardless of on-chain status** (a quarantined node can still record its upgrade); only versions that are both semver-valid AND PEP 440-parseable drive comparisons (e.g. `0.6.0-genlayer.1` is excluded — `packaging` can't parse it); it auto-creates a `TargetNodeVersion` when the fleet's highest STABLE release (bare `x.y.z`, no pre-release/build) exceeds the active target (`target_date=now`; an unparseable active target is never blindly superseded), writes each linked operator's `node_version_<network>` to their highest observed version via a direct `.update()` (bypassing `NodeVersionMixin.save()`'s path), and directly awards an already-approved `node-upgrade` Contribution (`_award_node_upgrade`, early-bonus 4/3/2/1) when a visible operator first reaches the active target. The per-operator loop is individually fault-isolated — one operator's failure never blocks the rest. Dedup shares the exact `version {v} [{network}]` notes key with the old manual flow so nothing double-awards.
- **Commands**: `validators/management/commands/rebuild_daily_snapshots.py` (`--days N`) re-materialises the daily rollup's observability columns from the raw observation log (preserves the on-chain `status`).
- **Views**: `validators/views.py`
- `/api/v1/validators/` - Validator profile CRUD for authenticated users
- `/api/v1/validators/me/` - GET/PATCH current validator profile
- `/api/v1/validators/me/` - GET current validator profile (read-only; PATCH removed — node versions are Grafana-sourced, not portal-editable)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
- `/api/v1/validators/wallets/` - Read-only validator wallet listing
- `/api/v1/validators/wallets/sync/` - POST cron-protected background sync trigger with DB-backed lock (on-chain validator sync)
- `/api/v1/validators/wallets/sync-grafana/` - POST cron-protected background sync trigger for Grafana observability cross-check (separate SyncLock row `grafana_status_sync` so it can run alongside the on-chain sync)
- `/api/v1/validators/wallets/wall-of-shame/` - Public read-only endpoint listing active validator wallets with `metrics_status` / `logs_status`. SHAME rows sort first. Cached 60s. Optional `?network=asimov|bradbury` filter.
- `/api/v1/validators/wallets/wall-of-shame/` - Public read-only endpoint listing active validator wallets with `metrics_status` / `logs_status`. SHAME rows sort first. Cached 60s. Optional `?network=asimov|bradbury` filter. Each wallet also carries `clean_streak_days` + `clean_streak_broken_by` (consecutive not-shamed days for that node, from `validators/streaks.py` over the daily rollup). The grouped `validators` output adds `network_streaks` — per-operator-per-network any-node-clean streaks (a network-day is clean if ≥1 of the operator's nodes was clean) — plus per-node `clean_streak_days` on each `networks` entry. Streaks start accumulating at deploy (history wasn't recorded before).
- `/api/v1/validators/wallets/grafana/` - Public minimal roster for the Grafana Infinity datasource (`GrafanaValidatorSerializer`). Flat array, one row per wallet across ALL statuses; fields: `network` (Grafana label value e.g. `asimov-phase5`), `node` (on-chain validator address == Prometheus `genlayer_node_info` `node` label, lowercased), `name`, `status`, `operator`, `account`/`account_name` (only for visible operators), `explorer_url`. Excludes observability/shame fields by design. Cached 60s. Optional `?network=asimov|bradbury` filter.

### Partners (Ecosystem Partners)
Expand Down Expand Up @@ -366,7 +383,7 @@ POST /api/auth/logout/
# Users
GET /api/v1/users/ (requires auth)
GET /api/v1/users/me/ (requires auth)
PATCH /api/v1/users/me/ (requires auth, only name)
PATCH /api/v1/users/me/ (requires auth; name/description/website/socials — node_version NOT editable, Grafana-sourced)
GET /api/v1/users/{address}/ (requires auth)
GET /api/v1/users/by-address/{address}/ (requires auth)
GET /api/v1/users/validators/ (requires auth)
Expand Down Expand Up @@ -484,6 +501,7 @@ Located in `.env` file:
- `GRAFANA_PROM_DS_UID` - Prometheus datasource UID (default `grafanacloud-prom`)
- `GRAFANA_LOKI_DS_UID` - Loki datasource UID (default `grafanacloud-logs`)
- `GRAFANA_ASIMOV_LABEL` / `GRAFANA_BRADBURY_LABEL` - Override the `network` label values Grafana queries use per testnet (defaults: `asimov-phase5`, `bradbury-phase1`)
- `NODE_VERSION_SHAME_GRACE_DAYS` - Grace period (days) after a target's `target_date` before a node still behind it is version-shamed, applied globally (default `3`)
- `SORSA_API_BASE_URL` - Sorsa API base URL (default `https://api.sorsa.io/v3`); used for Twitter follow verification in social_tasks and X follower counts in overview metrics.
- `SORSA_API_KEY` - Sorsa API key sent in the `ApiKey` header (secret, required). Store in AWS SSM (`/tally/{env}/sorsa_api_key`) for production.
- Note: the Sorsa request timeout and follow endpoint path are intentionally code constants in `social_tasks/sorsa_client.py`, not env vars. Changing the endpoint requires a code deploy anyway because the response parser lives in the same file.
Expand Down
5 changes: 5 additions & 0 deletions backend/tally/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,11 @@ def get_port_from_argv():
# Uptime lookback window (days) - how many days back to check for active status
UPTIME_LOOKBACK_DAYS = int(os.environ.get('UPTIME_LOOKBACK_DAYS', '7') or '7')

# Grace period (days) after a target's target_date before a node still running an
# older version is marked as version "shame". Applied globally at evaluation time
# (validators/version_status.py); changing it affects all targets, past and future.
NODE_VERSION_SHAME_GRACE_DAYS = int(os.environ.get('NODE_VERSION_SHAME_GRACE_DAYS', '3') or '3')

# AWS Health Check IPs - Allow these IPs to bypass ALLOWED_HOSTS
# Required environment variable with AWS internal/metadata service IPs
ALLOWED_CIDR_NETS = get_required_env('ALLOWED_CIDR_NETS').split(',')
Expand Down
34 changes: 4 additions & 30 deletions backend/users/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -305,19 +305,15 @@ def get_total_validators_count(self, obj):
class UserProfileUpdateSerializer(serializers.ModelSerializer):
"""
Serializer for updating user profile.
Allows updating name, profile fields, and validator node versions per network.
Allows updating name and profile fields. Node versions are NOT editable here:
they are sourced from Grafana (see validators/grafana_service.py) and only
displayed on the profile, never written from the portal.
"""
node_version_asimov = serializers.CharField(
required=False, allow_blank=True, allow_null=True, source='validator.node_version_asimov'
)
node_version_bradbury = serializers.CharField(
required=False, allow_blank=True, allow_null=True, source='validator.node_version_bradbury'
)
website = serializers.CharField(required=False, allow_blank=True, max_length=200)

class Meta:
model = User
fields = ['name', 'node_version_asimov', 'node_version_bradbury', 'description', 'website',
fields = ['name', 'description', 'website',
'telegram_handle', 'linkedin_handle']

def to_internal_value(self, data):
Expand Down Expand Up @@ -368,28 +364,6 @@ def validate_linkedin_handle(self, value):
value = value.split('?')[0]
return value

def update(self, instance, validated_data):
# Handle validator data if present
validator_data = validated_data.pop('validator', {})

# Update other user fields
for field, value in validated_data.items():
setattr(instance, field, value)

instance.save()

# Update or create validator if any node_version field is provided
if 'node_version_asimov' in validator_data or 'node_version_bradbury' in validator_data:
validator, created = Validator.objects.get_or_create(user=instance)
if 'node_version_asimov' in validator_data:
validator.node_version_asimov = validator_data['node_version_asimov']
if 'node_version_bradbury' in validator_data:
validator.node_version_bradbury = validator_data['node_version_bradbury']
validator.save()

return instance


class BuilderSerializer(serializers.ModelSerializer):
"""
Serializer for Builder profile.
Expand Down
Loading