Skip to content
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions ADR/ADR-001-Git-Branching-Model.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Architectural Decision Record 001: Git Branching Model

## Status
[<s>Proposed</s> | Accepted | <s>Deprecated</s> | <s>Superseded by [ADR-XXX]</s>]
<s>Proposed</s> | **Accepted** | <s>Deprecated</s> | <s>Superseded by [ADR-XXX]</s>

## Last Updated
2025-06-25
Expand Down Expand Up @@ -193,6 +193,10 @@ The reverse can also happen, where a repository using Git Flow can revert to Git

### Migration Path

> The authoritative, up-to-date operational migration procedure is maintained in
> [Guidelines/Branching.md](../Guidelines/Branching.md). The summary below records
> the intent at the time of this decision.

When upgrading from GitHub Flow to Git Flow:

* Create develop branch from current main
Expand All @@ -210,7 +214,7 @@ These decisions listed below were taken into account and they preexist this ADR.
* CI/CD pipeline configuration must support both GitHub Flow and Git Flow
* Code review exist for both branching models GitHub Flow and Git Flow
* Branch protection rules must be configured appropriately for each model, with main/master and develop branches protected for changes only by pull requests accepted by GateKeepers
* Deployment strategies may vary with the system or service and are not ncecessarily dependent on the chosen workflow, i.e. continuous vs. scheduled, where most ECMWF is deployed into operations with a scheduled release process.
* Deployment strategies may vary with the system or service and are not necessarily dependent on the chosen workflow, i.e. continuous vs. scheduled, where most ECMWF is deployed into operations with a scheduled release process.

## Consequences

Expand Down
2 changes: 1 addition & 1 deletion ADR/ADR-002-Approved-Dependency-CLI11.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Architectural Decision Record 002: Approved C++ Dependency CLI11

## Status
## Status

<s>Proposed</s> | **Accepted** | <s>Deprecated</s> | <s>Superseded by [ADR-XXX]</s>

Expand Down
2 changes: 1 addition & 1 deletion ADR/ADR-003-PyBind11-For-CPP-Bindings.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Status

[<s>Proposed</s> | **Accepted** | <s>Deprecated</s> | <s>Superseded by [ADR-XXX]</s>]
<s>Proposed</s> | **Accepted** | <s>Deprecated</s> | <s>Superseded by [ADR-XXX]</s>

## Last Updated

Expand Down
6 changes: 3 additions & 3 deletions ADR/ADR-004-Cascade-Features.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Architectural Decision Record 004: Building Cascade

## Status
[<s>Proposed</s> | **Accepted** | <s>Deprecated</s> | <s>Superseded by [ADR-XXX]</s>]
<s>Proposed</s> | **Accepted** | <s>Deprecated</s> | <s>Superseded by [ADR-XXX]</s>

## Last Updated
2025-07-28
Expand All @@ -14,7 +14,7 @@ This record captures the decision making of the initial feature set and design,

### Options Considered

Firstly, we considered using Dask as a comprehensive offering, and analyzed it in greater detail.
Firstly, we considered using Dask as a comprehensive offering, and analysed it in greater detail.
Other Dask-like solutions like Daft or Spark were not analysed thoroughly -- while they are presumably more performant than Dask in certain cases, the top level design and intent is very similar, and thus arguments again Dask apply equally.
Another possible option was Ray, which is has a less restrictive model than Dask, but based on a shallow analysis of its performance and ease of extensibility, we opted not to explore it further.

Expand Down Expand Up @@ -60,7 +60,7 @@ We purposefully don't implement a resource negotiator inside Cascade, as we don'
Instead, we rely on external one (currently Slurm, eventually Troika), and only expect a thin wrapper to invoke the resource negotiation and bootstrap Cascade processes.

For workflow executors, there is always the question whether the workflows themselves are aware of the executor, tightly coupled to it.
We chose a middle path -- we implement lightweight adapters (organized in a plugin fashion) for each class of workflow elements.
We chose a middle path -- we implement lightweight adapters (organised in a plugin fashion) for each class of workflow elements.
For example, individual Anemoi or PProc functions are oblivious of Cascade, but there exists Earthkit-Workflows-Anemoi and Earthkit-Workflows-Pproc which wrap anemoi runners or pproc pipelines in a Cascade-compatible fashion.
Those are exposed to the user in a fluent-style API.
This is additionally only thinly coupled to Cascade -- thus Cascade can execute workflows declared without fluent/plugin API, and vice versa we could implement a different executor for those fluent-declared objects.
Expand Down
4 changes: 2 additions & 2 deletions ADR/ADR-005-Storing-Of-ICON-Grid-Files.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Architectural Decision Record 004: Storing of MeteoSwiss ICON Grid Definition Files
# Architectural Decision Record 005: Storing of MeteoSwiss ICON Grid Definition Files

## Status
[<s>Proposed</s> | **Accepted** | <s>Deprecated</s> | <s>Superseded by [ADR-XXX]</s>]
<s>Proposed</s> | **Accepted** | <s>Deprecated</s> | <s>Superseded by [ADR-XXX]</s>

## Last Updated
2026-02-18
Expand Down
96 changes: 12 additions & 84 deletions ADR/ADR-006-MARS-Model-Keyword-for-DestinE-ODEDT.md
Original file line number Diff line number Diff line change
@@ -1,95 +1,23 @@
# Architectural Decision Record 006: MARS Model Keyword for DestinE ODEDT (DE330)

# Status
## Status

Withdrawn - Moved to become MARS Language Decision Record
<s>Proposed</s> | <s>Accepted</s> | <s>Deprecated</s> | **Withdrawn** — moved to [MARS-001](../MARS%20language/MARS-001-MARS-Model-Keyword-for-DestinE-ODEDT.md)

# Last Updated
## Last Updated

20.03.2026
2026-03-20

# Context
## Context

### Background
This decision was withdrawn from the ADR series and moved to the MARS Language
Decision Records, which govern changes to the MARS language. The proposal —
introducing a `model` keyword to the `on-demand-extremes-dt` data index for
Destination Earth (DE330) — is recorded in full, and kept up to date, in
[MARS-001: MARS Model Keyword for DestinE ODEDT](../MARS%20language/MARS-001-MARS-Model-Keyword-for-DestinE-ODEDT.md).
Refer to MARS-001 for the authoritative record.

The on-demand-extremes-dt routinely runs multiple models covering the same period, such as different NWP models (ALARO, AROME, HARMONIE-AROME) as well as several impact models (AQ, Hydrology, renewables, wildfire etc.).

Thus, an additional axis is needed to cope with several models running with all other mars metadata identical. To this end we would introduce the model keyword to the on-demand-extremes-dt data index, as has been introduced in other datasets and contexts such as class=ai. We would add the model keyword in the same way here. Thus this represents a consistent approach in-line with other datasets.

Values in ecCodes are currently as follows, but would be extended as the impact models become known and well formed ( `localConcepts/destine/modelNameConcept` ):

- IFS
- AIFS
- ICON
- ALARO
- AROME
- HARMONIE-AROME
- \<air-quality-impact-model\> (to be updated with value from DE330)
- \<hydrology-impact-model\> (to be updated with value from DE330)
- \<wildfire-impact-model\> (to be updated with value from DE330)
- …

### MARS archiving approach

MARS namespace:

<pre><code>{
"messages": [
{
"date": 20260211,
"time": 1200,
"expver": "0001",
"class": "d1",
"dataset": "on-demand-extremes-dt",
"georef": "u23er4",
"type": "fc",
"stream": "oper",
"step": 6,
"levtype": "sfc",
<strong>"model": "HARMONIE-AROME",</strong>
"timespan": "none",
"param": 167
}
]
}
</code></pre>

# Options Considered

### 1. Include model key

We believe this key is necessary to index data coming from different models within the on-demand Extremes DT project.

# Analysis

- Prototype at: [text](https://github.com/ecmwf/eccodes/tree/feature/ECC-2222-model-for-de330)
- This has been introduced in other datasets such as class=ai. We would add the model keyword in the same way here. Thus this represents a consistent approach in-line with other datasets.

# Decision

Meeting was held on 06/03/2026 to discuss the proposal with:

- Simon Smart
- Emanuele Danovaro
- Seb Villaume
- Robert Osinski
- Bojan Kasic

Following up meeting to confirm with User Services with:

- Paul Dando
- Bojan Kasic

Everyone was happy with the proposal. We will proceed with the implementation.

# Consequences

- ecCodes development to add model keyword to indexing namespace for on-demand-extremes-dt data
- metkit development to add new values of model keyword to language file.

# References

# Authors
## Authors

- Matthew Griffith
- Sebastien Villaume
Expand Down
6 changes: 3 additions & 3 deletions ADR/ADR-007-Approved-Dependency-libfmt.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ libfmt features (named arguments, custom join separators, terminal colours,
`fmt::memory_buffer`) have no standard equivalent at any horizon.

libfmt is the reference implementation from which `std::format` was
standardized. Its author, Victor Zverovich, is also the author and editor of the
standardised. Its author, Victor Zverovich, is also the author and editor of the
C++ formatting papers (P0645 for `std::format`, P2093 for `std::print`). The
libfmt API and the standard API are the same by design.

Expand Down Expand Up @@ -98,7 +98,7 @@ Defer until the codebase migrates to C++20, then use `std::format` directly.
Comparison](#feature-comparison-c20-vs-c23-vs-libfmt) — Tier 1). It does not
include `std::print` (C++23), range formatting (C++23), or features like
named arguments, custom join separators, and terminal colours (never
standardized).
standardised).
- Some `std::format` implementations have been slower and produced larger
binaries than libfmt. The gap is narrowing but was significant in early
GCC libstdc++ releases.
Expand Down Expand Up @@ -160,7 +160,7 @@ Effectively abandoned. Last meaningful update circa 2014. Not a viable option.

[libfmt](https://github.com/fmtlib/fmt)

The reference implementation that was standardized as `std::format` in C++20 and
The reference implementation that was standardised as `std::format` in C++20 and
`std::print` in C++23. Provides a type-safe, performant, and expressive
formatting API with features that go beyond any current or proposed C++ standard
revision.
Expand Down
2 changes: 1 addition & 1 deletion ADR/ADR-Guidelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ Before finalizing an ADR, ensure it meets these criteria:
- Use sequential numbering: ADR-001-Meaningful-Title, ADR-002-Another-Meaningful-Title, etc.
- Store ADRs in the central **Codex** repository under `/ADR/`
- Use descriptive filenames: `ADR-001-Git-Branching-Model.md`
- Use the [ADR index](./ADR-Index.md) to track all ADRs and their statuses
- Use the [ADR index](./README.md) to track all ADRs and their statuses

## Review Process

Expand Down
4 changes: 2 additions & 2 deletions ADR/ADR-Template.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Architectural Decision Record XXX: Title of the Decision

## Status
[**Proposed** | <s>Accepted</s> | <s>Deprecated</s> | <s>Superseded by [ADR-XXX]</s>]

**Proposed** | <s>Accepted</s> | <s>Deprecated</s> | <s>Superseded by [ADR-XXX]</s>

## Last Updated
YYYY-MM-DD
Expand Down
4 changes: 2 additions & 2 deletions ADR/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ Architectural Decision Records capturing key technical decisions for ECMWF softw
| [ADR-002 Approved Dependency CLI11](./ADR-002-Approved-Dependency-CLI11.md) | Accepted |
| [ADR-003 PyBind11 for C++ Bindings](./ADR-003-PyBind11-For-CPP-Bindings.md) | Accepted |
| [ADR-004 Cascade Features](./ADR-004-Cascade-Features.md) | Accepted |
| [ADR-005 Storing of ICON Grid Files](./ADR-005-Storing-Of-ICON-Grid-Files.md) | Accepted |
| [ADR-005 Storing of MeteoSwiss ICON Grid Definition Files](./ADR-005-Storing-Of-ICON-Grid-Files.md) | Accepted |
| [ADR-006 MARS Model Keyword for DestinE ODEDT](./ADR-006-MARS-Model-Keyword-for-DestinE-ODEDT.md) | Withdrawn — moved to [MARS-001](../MARS%20language/MARS-001-MARS-Model-Keyword-for-DestinE-ODEDT.md) |
| [ADR-007 Approved Dependency libfmt](./ADR-007-Approved-Dependency-libfmt.md) | Proposed |
| [ADR-007 Approved Dependency libfmt](./ADR-007-Approved-Dependency-libfmt.md) | Accepted |
| [ADR-008 From-Source Builds for C++ Stack Dependencies](./ADR-008-Stack-Dependencies.md) | Accepted |
| [ADR-009 Repository Audit Store](./ADR-009-Repository-Audit-Store.md) | Accepted |
| [ADR-010 Adopt SPDX Licence Identifiers and REUSE-Compliant File Headers](./ADR-010-SPDX-License-Identifiers-and-REUSE-Headers.md) | Accepted |
Expand Down
16 changes: 11 additions & 5 deletions Agent Skills/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Agent Skills

Reusable AI-agent skills that support ECMWF software governance processes.
Reusable AI-agent skills that support ECMWF software governance processes. These
skills are AI agents and operate under the
[AI Contributions to Software](../Guidelines/Ai-Contributions-To-Software.md)
guidelines: they keep a human in the loop, and they **report and gate rather
than act** — they never merge, fix, or publish on their own.

## Skills

Expand All @@ -10,11 +14,13 @@ Reusable AI-agent skills that support ECMWF software governance processes.
contents against the Codex open-sourcing guidance and common publication
risks, including licensing, README/maturity information, full-history secret
scanning, dependency licence review, git-history hygiene, and CI
configuration. Its mandatory security step is delegated to the `security-audit`
skill.
configuration. It prompts for a separate `security-audit` (required before
publication). A not-yet-run security audit is advisory; a security audit that
has been run and whose verdict is `NOT_READY` (i.e. it has open CRITICAL/HIGH
findings — see the `security-audit` skill) is a blocker.
- [`security-audit`](./security-audit/SKILL.md) — risk-tiered security audit of
a repository, run before publication (as the mandatory security step of the
open-source audit) and periodically afterwards. Builds a threat model,
a repository, run before publication (the security step of the open-source
audit) and periodically afterwards. Builds a threat model,
runs SAST / dependency / supply-chain tooling, reviews security-sensitive
surfaces, and for high-risk repositories adds adversarial testing and bounded
fuzzing. Produces a CWE-tagged pass/fail report.
Expand Down
2 changes: 2 additions & 0 deletions Contributing Upstream/Forking.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

External collaborators of ECMWF are expected to fork our repositories and then make a pull request from their fork. Likewise if we wish to contribute to other repositories, we often need to create a fork. This fork should exist in ECMWF GitHub but we want to take care that we distinguish it from code we own.

This page covers **ECMWF forking an external repository** (to prepare upstream contributions, or to maintain an ECMWF variant). The same mechanics appear in the "Fork" option for *inheriting* an external repository in [External Contributions](../Guidelines/External-Contributions.md); that section should point here for the standard notice template below.

## Process

1. Fork the repository. It will be public and cannot be changed.
Expand Down
11 changes: 10 additions & 1 deletion Contributing Upstream/README.md
Original file line number Diff line number Diff line change
@@ -1 +1,10 @@
- [Forking External Repositories](./Forking.md)
# Contributing to External Projects

Guidance for when ECMWF contributes to, or forks, repositories owned by other
organisations — for example upstreaming fixes, or maintaining an ECMWF variant of
an external project. For contributions *into* ECMWF repositories, see
[External Contributions](../Guidelines/External-Contributions.md).

## Contents

- [Forking External Repositories](./Forking.md) — how to fork an external repository into the ECMWF organisation and mark it clearly as a fork
12 changes: 6 additions & 6 deletions Documentation and Training/Documentation-Publishing-On-Sites.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,8 +249,8 @@ structure:
│ ├── pull_requests/ # Not Indexed
│ │ ├── PR-XXX/ # Will be deleted when PR is closed
│ │ └── PR-YYY/ # Will be deleted when PR is closed
│ ├── v1.2.3/ # Indexed through version.json
│ ├── v1.2.4/ # Indexed through version.json
│ ├── 1.2.3/ # Indexed through version.json
│ ├── 1.2.4/ # Indexed through version.json
│ ├── develop/ # Indexed through version.json
│ ├── latest/ # Indexed through version.json / Softlink to develop
│ ├── master/ # Indexed through version.json
Expand Down Expand Up @@ -435,7 +435,7 @@ regenerates the Sphinx version selector.
2. Uploads HTML to `<path>/<id>` via
[`ecmwf-sites-upload`](#ecmwf-sites-upload)
3. If `softlink` is provided, creates a server-side softlink via
`PUT` — e.g. `<path>/stable` -> `<path>/v2.1.0`. No content
`PUT` — e.g. `<path>/stable` -> `<path>/2.1.0`. No content
is duplicated.
4. Runs [`generate-version-selector`](#generate-version-selector)
to rebuild `version.json` for the Sphinx version dropdown
Expand All @@ -447,11 +447,11 @@ Versioned: https://sites.ecmwf.int/<space>/<name>/<path>/<id>/
Softlinked: https://sites.ecmwf.int/<space>/<name>/<path>/<softlink>/
```

Example with `path: fdb`, `id: v5.2.0`, `softlink: stable`:
Example with `path: fdb`, `id: 5.2.0`, `softlink: stable`:

```
.../docs/dev-section/fdb/v5.2.0/ <- concrete version
.../docs/dev-section/fdb/stable/ <- softlink to v5.2.0
.../docs/dev-section/fdb/5.2.0/ <- concrete version
.../docs/dev-section/fdb/stable/ <- softlink to 5.2.0
```

##### Inputs
Expand Down
8 changes: 2 additions & 6 deletions ESEE/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

The European Centre for Medium-Range Weather Forecasts (ECMWF) is developing the **ECMWF Software EnginE** (ESEE, pronounced "easy"), a flexible and interoperable software ecosystem designed for extreme-scale simulations, data fusion, data handling, data services, and machine learning.

ESEE is not a monolithic platform but a **set of opt-in components that work together seamlessly**, following a shared set of core data management principles. This modular approach allows users to integrate the parts that best suit their needs while ensuring interoperability across ECMWF services, Member and Cooperating States, and the broader Earth System community.
ESEE is not a monolithic platform but a **set of opt-in components that work together seamlessly**, following a shared set of core data management principles. This modular approach allows users to integrate the parts that best suit their needs while ensuring interoperability across ECMWF services, Member and Co-operating States, and the broader Earth System community.

ESEE provides the essential infrastructure needed to support cutting-edge scientific workflows at scale—delivering efficiency, adaptability, and performance where it matters most.

Expand All @@ -15,28 +15,24 @@ ESEE components are split into **four sub-categories**:

ESEE Production Chain encompasses components that drive the execution of operational and research workflows, managing complex dependencies, scheduling, and processing of data at scale.
Comment thread
tlmquintino marked this conversation as resolved.

[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/production_chain_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE)

## Data Provision

[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/data_provision_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE)

ESEE Data Provision category includes components responsible for managing, storing, and delivering data, ensuring that users can access large datasets with ease.
Comment thread
tlmquintino marked this conversation as resolved.

[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/data_provision_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE)

## User Interaction

[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/user_interaction_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE)
Comment thread
tlmquintino marked this conversation as resolved.
Outdated

ESEE User Interaction focuses on user APIs, visualization tools, and interactive environments that allow users to efficiently access and work with Earth System data and tools.
ESEE User Interaction focuses on user APIs, visualisation tools, and interactive environments that allow users to efficiently access and work with Earth System data and tools.

[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/user_interaction_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE)

## Foundation

[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/foundation_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE)

ESEE Foundation comprises essential libraries, frameworks, and utilities that ensure scalability, interoperability, and performance across all components.
Comment thread
tlmquintino marked this conversation as resolved.

[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/foundation_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE)
Loading