diff --git a/ADR/ADR-001-Git-Branching-Model.md b/ADR/ADR-001-Git-Branching-Model.md index a10f33f..f808c4d 100644 --- a/ADR/ADR-001-Git-Branching-Model.md +++ b/ADR/ADR-001-Git-Branching-Model.md @@ -1,7 +1,7 @@ # Architectural Decision Record 001: Git Branching Model ## Status -[Proposed | Accepted | Deprecated | Superseded by [ADR-XXX]] +Proposed | **Accepted** | Deprecated | Superseded by [ADR-XXX] ## Last Updated 2025-06-25 @@ -193,6 +193,10 @@ The reverse can also happen, where a repository using Git Flow can revert to Git ### Migration Path +> The authoritative, up-to-date operational migration procedure is maintained in +> [Guidelines/Branching.md](../Guidelines/Branching.md). The summary below records +> the intent at the time of this decision. + When upgrading from GitHub Flow to Git Flow: * Create develop branch from current main @@ -210,7 +214,7 @@ These decisions listed below were taken into account and they preexist this ADR. * CI/CD pipeline configuration must support both GitHub Flow and Git Flow * Code review exist for both branching models GitHub Flow and Git Flow * Branch protection rules must be configured appropriately for each model, with main/master and develop branches protected for changes only by pull requests accepted by GateKeepers -* Deployment strategies may vary with the system or service and are not ncecessarily dependent on the chosen workflow, i.e. continuous vs. scheduled, where most ECMWF is deployed into operations with a scheduled release process. +* Deployment strategies may vary with the system or service and are not necessarily dependent on the chosen workflow, i.e. continuous vs. scheduled, where most ECMWF is deployed into operations with a scheduled release process. ## Consequences diff --git a/ADR/ADR-002-Approved-Dependency-CLI11.md b/ADR/ADR-002-Approved-Dependency-CLI11.md index 311f093..28a8b29 100644 --- a/ADR/ADR-002-Approved-Dependency-CLI11.md +++ b/ADR/ADR-002-Approved-Dependency-CLI11.md @@ -1,6 +1,6 @@ # Architectural Decision Record 002: Approved C++ Dependency CLI11 - ## Status +## Status Proposed | **Accepted** | Deprecated | Superseded by [ADR-XXX] diff --git a/ADR/ADR-003-PyBind11-For-CPP-Bindings.md b/ADR/ADR-003-PyBind11-For-CPP-Bindings.md index 4c34a0b..e65d8f8 100644 --- a/ADR/ADR-003-PyBind11-For-CPP-Bindings.md +++ b/ADR/ADR-003-PyBind11-For-CPP-Bindings.md @@ -2,7 +2,7 @@ ## Status -[Proposed | **Accepted** | Deprecated | Superseded by [ADR-XXX]] +Proposed | **Accepted** | Deprecated | Superseded by [ADR-XXX] ## Last Updated diff --git a/ADR/ADR-004-Cascade-Features.md b/ADR/ADR-004-Cascade-Features.md index 5d9dce5..6652449 100644 --- a/ADR/ADR-004-Cascade-Features.md +++ b/ADR/ADR-004-Cascade-Features.md @@ -1,7 +1,7 @@ # Architectural Decision Record 004: Building Cascade ## Status -[Proposed | **Accepted** | Deprecated | Superseded by [ADR-XXX]] +Proposed | **Accepted** | Deprecated | Superseded by [ADR-XXX] ## Last Updated 2025-07-28 @@ -14,7 +14,7 @@ This record captures the decision making of the initial feature set and design, ### Options Considered -Firstly, we considered using Dask as a comprehensive offering, and analyzed it in greater detail. +Firstly, we considered using Dask as a comprehensive offering, and analysed it in greater detail. Other Dask-like solutions like Daft or Spark were not analysed thoroughly -- while they are presumably more performant than Dask in certain cases, the top level design and intent is very similar, and thus arguments again Dask apply equally. Another possible option was Ray, which is has a less restrictive model than Dask, but based on a shallow analysis of its performance and ease of extensibility, we opted not to explore it further. @@ -60,7 +60,7 @@ We purposefully don't implement a resource negotiator inside Cascade, as we don' Instead, we rely on external one (currently Slurm, eventually Troika), and only expect a thin wrapper to invoke the resource negotiation and bootstrap Cascade processes. For workflow executors, there is always the question whether the workflows themselves are aware of the executor, tightly coupled to it. -We chose a middle path -- we implement lightweight adapters (organized in a plugin fashion) for each class of workflow elements. +We chose a middle path -- we implement lightweight adapters (organised in a plugin fashion) for each class of workflow elements. For example, individual Anemoi or PProc functions are oblivious of Cascade, but there exists Earthkit-Workflows-Anemoi and Earthkit-Workflows-Pproc which wrap anemoi runners or pproc pipelines in a Cascade-compatible fashion. Those are exposed to the user in a fluent-style API. This is additionally only thinly coupled to Cascade -- thus Cascade can execute workflows declared without fluent/plugin API, and vice versa we could implement a different executor for those fluent-declared objects. diff --git a/ADR/ADR-005-Storing-Of-ICON-Grid-Files.md b/ADR/ADR-005-Storing-Of-ICON-Grid-Files.md index e2e78ee..f359588 100644 --- a/ADR/ADR-005-Storing-Of-ICON-Grid-Files.md +++ b/ADR/ADR-005-Storing-Of-ICON-Grid-Files.md @@ -1,7 +1,7 @@ -# Architectural Decision Record 004: Storing of MeteoSwiss ICON Grid Definition Files +# Architectural Decision Record 005: Storing of MeteoSwiss ICON Grid Definition Files ## Status -[Proposed | **Accepted** | Deprecated | Superseded by [ADR-XXX]] +Proposed | **Accepted** | Deprecated | Superseded by [ADR-XXX] ## Last Updated 2026-02-18 diff --git a/ADR/ADR-006-MARS-Model-Keyword-for-DestinE-ODEDT.md b/ADR/ADR-006-MARS-Model-Keyword-for-DestinE-ODEDT.md index e7efa2d..d670a97 100644 --- a/ADR/ADR-006-MARS-Model-Keyword-for-DestinE-ODEDT.md +++ b/ADR/ADR-006-MARS-Model-Keyword-for-DestinE-ODEDT.md @@ -1,95 +1,23 @@ # Architectural Decision Record 006: MARS Model Keyword for DestinE ODEDT (DE330) -# Status +## Status -Withdrawn - Moved to become MARS Language Decision Record +Proposed | Accepted | Deprecated | **Withdrawn** — moved to [MARS-001](../MARS%20language/MARS-001-MARS-Model-Keyword-for-DestinE-ODEDT.md) -# Last Updated +## Last Updated -20.03.2026 +2026-03-20 -# Context +## Context -### Background +This decision was withdrawn from the ADR series and moved to the MARS Language +Decision Records, which govern changes to the MARS language. The proposal — +introducing a `model` keyword to the `on-demand-extremes-dt` data index for +Destination Earth (DE330) — is recorded in full, and kept up to date, in +[MARS-001: MARS Model Keyword for DestinE ODEDT](../MARS%20language/MARS-001-MARS-Model-Keyword-for-DestinE-ODEDT.md). +Refer to MARS-001 for the authoritative record. -The on-demand-extremes-dt routinely runs multiple models covering the same period, such as different NWP models (ALARO, AROME, HARMONIE-AROME) as well as several impact models (AQ, Hydrology, renewables, wildfire etc.). - -Thus, an additional axis is needed to cope with several models running with all other mars metadata identical. To this end we would introduce the model keyword to the on-demand-extremes-dt data index, as has been introduced in other datasets and contexts such as class=ai. We would add the model keyword in the same way here. Thus this represents a consistent approach in-line with other datasets. - -Values in ecCodes are currently as follows, but would be extended as the impact models become known and well formed ( `localConcepts/destine/modelNameConcept` ): - -- IFS -- AIFS -- ICON -- ALARO -- AROME -- HARMONIE-AROME -- \ (to be updated with value from DE330) -- \ (to be updated with value from DE330) -- \ (to be updated with value from DE330) -- … - -### MARS archiving approach - -MARS namespace: - -
{
-  "messages": [
-    {
-      "date": 20260211,
-      "time": 1200,
-      "expver": "0001",
-      "class": "d1",
-      "dataset": "on-demand-extremes-dt",
-      "georef": "u23er4",
-      "type": "fc",
-      "stream": "oper",
-      "step": 6,
-      "levtype": "sfc",
-      "model": "HARMONIE-AROME",
-      "timespan": "none",
-      "param": 167
-    }
-  ]
-}
-
- -# Options Considered - -### 1. Include model key - -We believe this key is necessary to index data coming from different models within the on-demand Extremes DT project. - -# Analysis - -- Prototype at: [text](https://github.com/ecmwf/eccodes/tree/feature/ECC-2222-model-for-de330) -- This has been introduced in other datasets such as class=ai. We would add the model keyword in the same way here. Thus this represents a consistent approach in-line with other datasets. - -# Decision - -Meeting was held on 06/03/2026 to discuss the proposal with: - -- Simon Smart -- Emanuele Danovaro -- Seb Villaume -- Robert Osinski -- Bojan Kasic - -Following up meeting to confirm with User Services with: - -- Paul Dando -- Bojan Kasic - -Everyone was happy with the proposal. We will proceed with the implementation. - -# Consequences - -- ecCodes development to add model keyword to indexing namespace for on-demand-extremes-dt data -- metkit development to add new values of model keyword to language file. - -# References - -# Authors +## Authors - Matthew Griffith - Sebastien Villaume diff --git a/ADR/ADR-007-Approved-Dependency-libfmt.md b/ADR/ADR-007-Approved-Dependency-libfmt.md index 3b76ddb..7692346 100644 --- a/ADR/ADR-007-Approved-Dependency-libfmt.md +++ b/ADR/ADR-007-Approved-Dependency-libfmt.md @@ -45,7 +45,7 @@ libfmt features (named arguments, custom join separators, terminal colours, `fmt::memory_buffer`) have no standard equivalent at any horizon. libfmt is the reference implementation from which `std::format` was -standardized. Its author, Victor Zverovich, is also the author and editor of the +standardised. Its author, Victor Zverovich, is also the author and editor of the C++ formatting papers (P0645 for `std::format`, P2093 for `std::print`). The libfmt API and the standard API are the same by design. @@ -98,7 +98,7 @@ Defer until the codebase migrates to C++20, then use `std::format` directly. Comparison](#feature-comparison-c20-vs-c23-vs-libfmt) — Tier 1). It does not include `std::print` (C++23), range formatting (C++23), or features like named arguments, custom join separators, and terminal colours (never - standardized). + standardised). - Some `std::format` implementations have been slower and produced larger binaries than libfmt. The gap is narrowing but was significant in early GCC libstdc++ releases. @@ -160,7 +160,7 @@ Effectively abandoned. Last meaningful update circa 2014. Not a viable option. [libfmt](https://github.com/fmtlib/fmt) -The reference implementation that was standardized as `std::format` in C++20 and +The reference implementation that was standardised as `std::format` in C++20 and `std::print` in C++23. Provides a type-safe, performant, and expressive formatting API with features that go beyond any current or proposed C++ standard revision. diff --git a/ADR/ADR-Guidelines.md b/ADR/ADR-Guidelines.md index 94e0a46..1ce91c6 100644 --- a/ADR/ADR-Guidelines.md +++ b/ADR/ADR-Guidelines.md @@ -129,7 +129,7 @@ Before finalizing an ADR, ensure it meets these criteria: - Use sequential numbering: ADR-001-Meaningful-Title, ADR-002-Another-Meaningful-Title, etc. - Store ADRs in the central **Codex** repository under `/ADR/` - Use descriptive filenames: `ADR-001-Git-Branching-Model.md` -- Use the [ADR index](./ADR-Index.md) to track all ADRs and their statuses +- Use the [ADR index](./README.md) to track all ADRs and their statuses ## Review Process diff --git a/ADR/ADR-Template.md b/ADR/ADR-Template.md index 9a77c77..eb3501a 100644 --- a/ADR/ADR-Template.md +++ b/ADR/ADR-Template.md @@ -1,7 +1,7 @@ # Architectural Decision Record XXX: Title of the Decision - ## Status -[**Proposed** | Accepted | Deprecated | Superseded by [ADR-XXX]] + +**Proposed** | Accepted | Deprecated | Superseded by [ADR-XXX] ## Last Updated YYYY-MM-DD diff --git a/ADR/README.md b/ADR/README.md index 8293aab..769ccc6 100644 --- a/ADR/README.md +++ b/ADR/README.md @@ -15,9 +15,9 @@ Architectural Decision Records capturing key technical decisions for ECMWF softw | [ADR-002 Approved Dependency CLI11](./ADR-002-Approved-Dependency-CLI11.md) | Accepted | | [ADR-003 PyBind11 for C++ Bindings](./ADR-003-PyBind11-For-CPP-Bindings.md) | Accepted | | [ADR-004 Cascade Features](./ADR-004-Cascade-Features.md) | Accepted | -| [ADR-005 Storing of ICON Grid Files](./ADR-005-Storing-Of-ICON-Grid-Files.md) | Accepted | +| [ADR-005 Storing of MeteoSwiss ICON Grid Definition Files](./ADR-005-Storing-Of-ICON-Grid-Files.md) | Accepted | | [ADR-006 MARS Model Keyword for DestinE ODEDT](./ADR-006-MARS-Model-Keyword-for-DestinE-ODEDT.md) | Withdrawn — moved to [MARS-001](../MARS%20language/MARS-001-MARS-Model-Keyword-for-DestinE-ODEDT.md) | -| [ADR-007 Approved Dependency libfmt](./ADR-007-Approved-Dependency-libfmt.md) | Proposed | +| [ADR-007 Approved Dependency libfmt](./ADR-007-Approved-Dependency-libfmt.md) | Accepted | | [ADR-008 From-Source Builds for C++ Stack Dependencies](./ADR-008-Stack-Dependencies.md) | Accepted | | [ADR-009 Repository Audit Store](./ADR-009-Repository-Audit-Store.md) | Accepted | | [ADR-010 Adopt SPDX Licence Identifiers and REUSE-Compliant File Headers](./ADR-010-SPDX-License-Identifiers-and-REUSE-Headers.md) | Accepted | diff --git a/Agent Skills/README.md b/Agent Skills/README.md index e86f475..88b20a2 100644 --- a/Agent Skills/README.md +++ b/Agent Skills/README.md @@ -1,6 +1,10 @@ # Agent Skills -Reusable AI-agent skills that support ECMWF software governance processes. +Reusable AI-agent skills that support ECMWF software governance processes. These +skills are AI agents and operate under the +[AI Contributions to Software](../Guidelines/Ai-Contributions-To-Software.md) +guidelines: they keep a human in the loop, and they **report and gate rather +than act** — they never merge, fix, or publish on their own. ## Skills @@ -10,11 +14,13 @@ Reusable AI-agent skills that support ECMWF software governance processes. contents against the Codex open-sourcing guidance and common publication risks, including licensing, README/maturity information, full-history secret scanning, dependency licence review, git-history hygiene, and CI - configuration. Its mandatory security step is delegated to the `security-audit` - skill. + configuration. It prompts for a separate `security-audit` (required before + publication). A not-yet-run security audit is advisory; a security audit that + has been run and whose verdict is `NOT_READY` (i.e. it has open CRITICAL/HIGH + findings — see the `security-audit` skill) is a blocker. - [`security-audit`](./security-audit/SKILL.md) — risk-tiered security audit of - a repository, run before publication (as the mandatory security step of the - open-source audit) and periodically afterwards. Builds a threat model, + a repository, run before publication (the security step of the open-source + audit) and periodically afterwards. Builds a threat model, runs SAST / dependency / supply-chain tooling, reviews security-sensitive surfaces, and for high-risk repositories adds adversarial testing and bounded fuzzing. Produces a CWE-tagged pass/fail report. diff --git a/Contributing Upstream/Forking.md b/Contributing Upstream/Forking.md index 8e16e8f..64fea4f 100644 --- a/Contributing Upstream/Forking.md +++ b/Contributing Upstream/Forking.md @@ -2,6 +2,8 @@ External collaborators of ECMWF are expected to fork our repositories and then make a pull request from their fork. Likewise if we wish to contribute to other repositories, we often need to create a fork. This fork should exist in ECMWF GitHub but we want to take care that we distinguish it from code we own. +This page covers **ECMWF forking an external repository** (to prepare upstream contributions, or to maintain an ECMWF variant). The same mechanics appear in the "Fork" option for *inheriting* an external repository in [External Contributions](../Guidelines/External-Contributions.md); that section should point here for the standard notice template below. + ## Process 1. Fork the repository. It will be public and cannot be changed. diff --git a/Contributing Upstream/README.md b/Contributing Upstream/README.md index 0975cc9..e04a0a8 100644 --- a/Contributing Upstream/README.md +++ b/Contributing Upstream/README.md @@ -1 +1,10 @@ -- [Forking External Repositories](./Forking.md) \ No newline at end of file +# Contributing to External Projects + +Guidance for when ECMWF contributes to, or forks, repositories owned by other +organisations — for example upstreaming fixes, or maintaining an ECMWF variant of +an external project. For contributions *into* ECMWF repositories, see +[External Contributions](../Guidelines/External-Contributions.md). + +## Contents + +- [Forking External Repositories](./Forking.md) — how to fork an external repository into the ECMWF organisation and mark it clearly as a fork diff --git a/Documentation and Training/Documentation-Publishing-On-Sites.md b/Documentation and Training/Documentation-Publishing-On-Sites.md index cef07d2..41de5e6 100644 --- a/Documentation and Training/Documentation-Publishing-On-Sites.md +++ b/Documentation and Training/Documentation-Publishing-On-Sites.md @@ -249,8 +249,8 @@ structure: │ ├── pull_requests/ # Not Indexed │ │ ├── PR-XXX/ # Will be deleted when PR is closed │ │ └── PR-YYY/ # Will be deleted when PR is closed -│ ├── v1.2.3/ # Indexed through version.json -│ ├── v1.2.4/ # Indexed through version.json +│ ├── 1.2.3/ # Indexed through version.json +│ ├── 1.2.4/ # Indexed through version.json │ ├── develop/ # Indexed through version.json │ ├── latest/ # Indexed through version.json / Softlink to develop │ ├── master/ # Indexed through version.json @@ -435,7 +435,7 @@ regenerates the Sphinx version selector. 2. Uploads HTML to `/` via [`ecmwf-sites-upload`](#ecmwf-sites-upload) 3. If `softlink` is provided, creates a server-side softlink via - `PUT` — e.g. `/stable` -> `/v2.1.0`. No content + `PUT` — e.g. `/stable` -> `/2.1.0`. No content is duplicated. 4. Runs [`generate-version-selector`](#generate-version-selector) to rebuild `version.json` for the Sphinx version dropdown @@ -447,11 +447,11 @@ Versioned: https://sites.ecmwf.int///// Softlinked: https://sites.ecmwf.int///// ``` -Example with `path: fdb`, `id: v5.2.0`, `softlink: stable`: +Example with `path: fdb`, `id: 5.2.0`, `softlink: stable`: ``` -.../docs/dev-section/fdb/v5.2.0/ <- concrete version -.../docs/dev-section/fdb/stable/ <- softlink to v5.2.0 +.../docs/dev-section/fdb/5.2.0/ <- concrete version +.../docs/dev-section/fdb/stable/ <- softlink to 5.2.0 ``` ##### Inputs diff --git a/ESEE/README.md b/ESEE/README.md index 3e2d762..ae84fcc 100644 --- a/ESEE/README.md +++ b/ESEE/README.md @@ -3,7 +3,7 @@ The European Centre for Medium-Range Weather Forecasts (ECMWF) is developing the **ECMWF Software EnginE** (ESEE, pronounced "easy"), a flexible and interoperable software ecosystem designed for extreme-scale simulations, data fusion, data handling, data services, and machine learning. -ESEE is not a monolithic platform but a **set of opt-in components that work together seamlessly**, following a shared set of core data management principles. This modular approach allows users to integrate the parts that best suit their needs while ensuring interoperability across ECMWF services, Member and Cooperating States, and the broader Earth System community. +ESEE is not a monolithic platform but a **set of opt-in components that work together seamlessly**, following a shared set of core data management principles. This modular approach allows users to integrate the parts that best suit their needs while ensuring interoperability across ECMWF services, Member and Co-operating States, and the broader Earth System community. ESEE provides the essential infrastructure needed to support cutting-edge scientific workflows at scale—delivering efficiency, adaptability, and performance where it matters most. @@ -11,32 +11,28 @@ ESEE components are split into **four sub-categories**: ## Production Chain -[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/production_chain_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE) +[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/production_chain_badge.svg)](https://github.com/ecmwf/codex/tree/main/ESEE) ESEE Production Chain encompasses components that drive the execution of operational and research workflows, managing complex dependencies, scheduling, and processing of data at scale. - [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/production_chain_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE) ## Data Provision -[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/data_provision_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE) +[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/data_provision_badge.svg)](https://github.com/ecmwf/codex/tree/main/ESEE) ESEE Data Provision category includes components responsible for managing, storing, and delivering data, ensuring that users can access large datasets with ease. - [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/data_provision_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE) ## User Interaction -[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/user_interaction_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE) +[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/user_interaction_badge.svg)](https://github.com/ecmwf/codex/tree/main/ESEE) -ESEE User Interaction focuses on user APIs, visualization tools, and interactive environments that allow users to efficiently access and work with Earth System data and tools. +ESEE User Interaction focuses on user APIs, visualisation tools, and interactive environments that allow users to efficiently access and work with Earth System data and tools. - [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/user_interaction_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE) ## Foundation -[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/foundation_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE) +[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/foundation_badge.svg)](https://github.com/ecmwf/codex/tree/main/ESEE) ESEE Foundation comprises essential libraries, frameworks, and utilities that ensure scalability, interoperability, and performance across all components. - [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE/foundation_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/ESEE) diff --git a/Guidelines/Ai-Contributions-To-Software.md b/Guidelines/Ai-Contributions-To-Software.md index 1997c39..ab2023d 100644 --- a/Guidelines/Ai-Contributions-To-Software.md +++ b/Guidelines/Ai-Contributions-To-Software.md @@ -19,7 +19,7 @@ At ECMWF, we view AI Code Agents and Large Language Models (LLMs) as powerful pr ### Code Review Protocols * **Mandatory Review:** Code generated by AI Agents is always subject to review, as customary for any code contribution. This should include both automated checks and human peer review. This code review may itself be assisted by AI agents, but the final review and approval must be performed by a human reviewer. -* **Reviews by AI Agents:** If you are using an AI agent to support code reviews, a good practice is to have a different AI agent model review the code, to provide a form of "cross-checking" and reduce the risk of blind spots or biases in a single model. +* **Reviews by AI Agents:** If you are using an AI agent to support code reviews, a good practice is to have a different AI agent model review the code, to provide a form of "cross-checking" and reduce the risk of blind spots or biases in a single model. ECMWF ships reusable governance agents for this purpose — the [Agent Skills](../Agent%20Skills/README.md) (e.g. `open-source-audit`, `security-audit`), which report and gate but never merge or publish on their own. * **Peer Reviewing:** AI-generated code should be reviewed by a human *other* than the developer who prompted or controlled the AI agent. We recognise this may not always be possible, but it should be the preferred approach. * **Operational Note:** This independent peer-review approach becomes even more important, preferred and highly recommended for any code entering operational, critical, or heavily relied-upon execution paths. @@ -39,4 +39,4 @@ At ECMWF, we view AI Code Agents and Large Language Models (LLMs) as powerful pr * **Warning on Intellectual Property and Licensing:** Note that AI models can occasionally reproduce snippets of copyrighted code (e.g., strictly GPL-licensed code). Developers should be reasonably confident the generated code does not violate ECMWF's open-source licensing strategy (mostly Apache 2.0). -* **Test Coverage:** As for all external contributions, it is highly desirable that AI-generated code should be accompanied by unit tests. However, it is more important for AI generated code that *some* human-verified unit tests are included. AI is notoriously good at writing code that *looks* right but fails edge cases, making strict testing coverage even more important. +* **Test Coverage:** AI-generated code must meet the same testing requirement as any other contribution (see [External Contributions](./External-Contributions.md)); it is **not** exempt. It is especially important that AI-generated code carries *human-verified* unit tests — AI is notoriously good at writing code that *looks* right but fails edge cases, so human-checked tests matter even more here. diff --git a/Guidelines/Branching.md b/Guidelines/Branching.md index 6891dc0..cf6de6e 100644 --- a/Guidelines/Branching.md +++ b/Guidelines/Branching.md @@ -1,6 +1,6 @@ # Branching Patterns -This document describes the branching patterns used across ECMWF repositories. The choice of pattern for a given repository is governed by [ADR-001: Git Branching Model](../ADR/ADR-001-Git-Branching-Model.md). +This document describes the branching patterns used across ECMWF repositories. [ADR-001: Git Branching Model](../ADR/ADR-001-Git-Branching-Model.md) records the *decision and rationale*; this document is the *normative operational procedure*. Version tags follow Semantic Versioning in the clean `x.y.z` form (no `v` prefix) — see the canonical tag and prerelease (`x.y.z-upstream.N`) conventions in [External Contributions](./External-Contributions.md). ## Contents @@ -92,7 +92,7 @@ For critical production issues: ``` 2. Apply the fix and push. 3. Open a PR against `main`; follow the expedited review process agreed by your team. -4. After merge, tag a new patch release on `main` (`x.y.z+1`). +4. After merge, tag the next patch release on `main` (e.g. `1.4.3` → `1.4.4`). --- @@ -187,7 +187,7 @@ when users cannot immediately upgrade to the latest version: 1. **Create a support branch** from the tag of the release to be maintained: ``` - git switch -c support/1.x v1.9.0 + git switch -c support/1.x 1.9.0 ``` 2. **Apply fixes** — cherry-pick or develop fixes on the support branch. Only critical bug fixes and security patches should be included. diff --git a/Guidelines/Containerisation.md b/Guidelines/Containerisation.md index 153588a..ea66bfd 100644 --- a/Guidelines/Containerisation.md +++ b/Guidelines/Containerisation.md @@ -1,13 +1,13 @@ -# Best Practices for Containerization +# Best Practices for Containerisation ## Table of Contents 1. [Introduction](#1-introduction) - [The 12-Factor App](#the-12-factor-app) -2. [Why Containerize?](#2-why-containerize) +2. [Why Containerise?](#2-why-containerise) - [Declarative & Automated Setup](#21-declarative--automated-setup) - [Clean Contract with the Operating System](#22-clean-contract-with-the-operating-system) - - [Standardized Environments](#standardized-environments) + - [Standardised Environments](#standardised-environments) - [Automation-Friendly](#automation-friendly) - [It is Lightweight](#23-it-is-lightweight) - [No Virtualization Overhead](#no-virtualization-overhead) @@ -27,7 +27,7 @@ - [Operational Complexity](#operational-complexity) - [Why Use Kubernetes Then?](#43-why-use-kubernetes-then) - [Easy Scaling](#easy-scaling) - - [Efficient Resource Utilization](#efficient-resource-utilization) + - [Efficient Resource Utilisation](#efficient-resource-utilisation) - [Reduced Dependence on System Administrators](#reduced-dependence-on-system-administrators) - [Multi-Application Management](#multi-application-management) 5. [Tools](#5-tools) @@ -44,7 +44,7 @@ --- ## 1. Introduction: -Before discussing the benefits of containerization, it might be important to understand one of a key foundation for modern, cloud-native applications—[the 12-Factor App](https://12factor.net/). +Before discussing the benefits of containerisation, it might be important to understand one of the key foundations for modern, cloud-native applications—[the 12-Factor App](https://12factor.net/). ### The 12-Factor App @@ -60,18 +60,18 @@ Adhering to these principles bridges the gap between development and production --- -## 2. Why Containerize? +## 2. Why Containerise? -Containerization packages an application along with all its dependencies into a lightweight, self-contained unit. This approach brings several advantages that complement the 12-Factor principles. +Containerisation packages an application along with all its dependencies into a lightweight, self-contained unit. This approach brings several advantages that complement the 12-Factor principles. ### 2.1 Declarative & Automated Setup -By defining environments and configurations declaratively (e.g., Dockerfiles, Helm charts), containerization standardizes the setup process. This minimizes onboarding time and ensures reproducibility across different environments. +By defining environments and configurations declaratively (e.g., Dockerfiles, Helm charts), containerisation standardises the setup process. This minimises onboarding time and ensures reproducibility across different environments. **No more "it works on my computer."** ### 2.2 Clean Contract with the Operating System -#### Standardized Environments +#### Standardised Environments Containers encapsulate applications and their dependencies while relying on a shared host kernel. This ensures portability across diverse execution environments without infrastructure-specific configurations. #### Automation-Friendly @@ -95,13 +95,13 @@ Containers are designed to be ephemeral; each instance starts fresh with no resi #### Effortless Horizontal Scaling Containers are stateless and lightweight, making them ideal for scaling workloads. Orchestration tools such as **Kubernetes** facilitate automatic replication of containers to handle fluctuating traffic. -Instead of upgrading a single large server (vertical scaling), containerized applications scale by increasing the number of running instances (horizontal scaling). This ensures: +Instead of upgrading a single large server (vertical scaling), containerised applications scale by increasing the number of running instances (horizontal scaling). This ensures: - No single point of failure—if one container crashes, others continue processing requests. - On-demand scaling—applications dynamically scale up or down based on real-time load. -- Efficient resource utilization—only necessary instances run at any given time. +- Efficient resource utilisation—only necessary instances run at any given time. #### Orchestration: The Magic Behind Scaling -Scaling manually is inefficient. Modern orchestration platforms automate scaling based on traffic, resource utilization, and predefined policies. +Scaling manually is inefficient. Modern orchestration platforms automate scaling based on traffic, resource utilisation, and predefined policies. **Key benefits include:** - **Auto-scaling:** Adjusts the number of running containers dynamically. @@ -134,7 +134,7 @@ Virtual machines operate like separate apartments with individual utilities, whi ## 4. Why Kubernetes? -Kubernetes is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. It abstracts away the complexity of infrastructure management, enabling teams to focus on building and running applications efficiently. +Kubernetes is an open-source container orchestration platform that automates the deployment, scaling, and management of containerised applications. It abstracts away the complexity of infrastructure management, enabling teams to focus on building and running applications efficiently. ### 4.1 What Is Kubernetes? At its core, Kubernetes provides a framework to run distributed systems resiliently. It handles tasks such as scaling, failover, deployment rollouts, and service discovery, ensuring applications remain highly available and performant. Kubernetes operates on a cluster of machines (nodes), managing containers grouped into pods—the smallest deployable units in Kubernetes. Key features include: @@ -145,7 +145,7 @@ At its core, Kubernetes provides a framework to run distributed systems resilien * **Multi-cloud Support:** Kubernetes provides a consistent deployment experience across on-premises data centers and public clouds. ### 4.2 What’s the Cost? -While Kubernetes offers tremendous benefits, it comes with some upfront costs that organization must consider: +While Kubernetes offers tremendous benefits, it comes with some upfront costs that organisation must consider: #### Learning Cost Kubernetes introduces a steep learning curve due to its extensive ecosystem and concepts such as pods, services, deployments, ingress controllers, and networking policies. Teams must invest time in mastering these components as well as tools like Helm (for package management) and kubectl (for cluster interaction). @@ -154,16 +154,16 @@ Kubernetes introduces a steep learning curve due to its extensive ecosystem and Setting up a project in Kubernetes involves creating configuration files (manifests) or Helm charts to define application deployments, services, secrets, and other resources. While this adds initial complexity, these configurations are reusable and modular. Once established, they significantly streamline future deployments and updates. #### Operational Complexity -Managing a Kubernetes cluster requires additional expertise in areas such as monitoring (e.g., Prometheus), logging (e.g., Fluentd, OpenTelemetry), security policies, and resource optimization. +Managing a Kubernetes cluster requires additional expertise in areas such as monitoring (e.g., Prometheus), logging (e.g., Fluentd, OpenTelemetry), security policies, and resource optimisation. ### 4.3 Why Use Kubernetes Then? Despite the costs, the advantages of Kubernetes far outweigh its challenges: #### Easy Scaling -Kubernetes makes horizontal scaling effortless by allowing you to add or remove container instances dynamically based on traffic or resource utilization. With auto-scaling features like the Horizontal Pod Autoscaler (HPA), applications can handle sudden spikes in demand without manual intervention. +Kubernetes makes horizontal scaling effortless by allowing you to add or remove container instances dynamically based on traffic or resource utilisation. With auto-scaling features like the Horizontal Pod Autoscaler (HPA), applications can handle sudden spikes in demand without manual intervention. -#### Efficient Resource Utilization -Kubernetes optimizes resource allocation by scheduling workloads intelligently across nodes based on CPU and memory requirements. This ensures that hardware is used efficiently while avoiding over-provisioning or underutilization. +#### Efficient Resource Utilisation +Kubernetes optimises resource allocation by scheduling workloads intelligently across nodes based on CPU and memory requirements. This ensures that hardware is used efficiently while avoiding over-provisioning or underutilisation. #### Reduced Dependence on System Administrators By automating routine tasks such as rollouts, rollbacks, health checks, and failover mechanisms, Kubernetes reduces reliance on system administrators for day-to-day operations. Developers can deploy applications independently using declarative configurations without needing deep infrastructure knowledge. @@ -177,7 +177,7 @@ Ultimately, Kubernetes excels in environments where scalability, portability, an ## 5. Tools -The container ecosystem is rich with tools designed to simplify development, deployment, debugging, and monitoring. While many options exist, this is a **curated list** of tools that stand out for their practicality and reliability in containerized workflows. +The container ecosystem is rich with tools designed to simplify development, deployment, debugging, and monitoring. While many options exist, this is a **curated list** of tools that stand out for their practicality and reliability in containerised workflows. ### 5.1 Skaffold @@ -225,13 +225,13 @@ By using Telepresence, teams can debug services in real-time without disrupting - **Flexible Filtering**: Supports regex-based filtering to display only relevant log messages. - **Compatibility**: Works seamlessly with Kubernetes and can be integrated into debugging workflows. -Stern provides a **real-time** view of application behavior, making it indispensable for troubleshooting and monitoring large-scale deployments. +Stern provides a **real-time** view of application behaviour, making it indispensable for troubleshooting and monitoring large-scale deployments. --- ## 6. Repository Structure -A well-organized repository structure ensures that your containerized application can be built, tested, and deployed consistently across various environments. An increasingly popular best practice is **splitting your Helm charts into a separate public repository** and placing environment-specific configurations (including overrides and sensitive data) in a **private configuration repository**. This approach leverages hierarchical Helm configurations, allowing you to manage multiple deployments from a single source of truth while keeping sensitive details out of public view. +A well-organised repository structure ensures that your containerised application can be built, tested, and deployed consistently across various environments. An increasingly popular best practice is **splitting your Helm charts into a separate public repository** and placing environment-specific configurations (including overrides and sensitive data) in a **private configuration repository**. This approach leverages hierarchical Helm configurations, allowing you to manage multiple deployments from a single source of truth while keeping sensitive details out of public view. ### 6.1 Public Helm Chart Repository diff --git a/Guidelines/External-Contributions.md b/Guidelines/External-Contributions.md index 8eb30e7..22c51d6 100644 --- a/Guidelines/External-Contributions.md +++ b/Guidelines/External-Contributions.md @@ -59,8 +59,8 @@ If ECMWF inherits an external repository (e.g., a Code4Earth project), one of th - Adding the CLA agreement to the PR template. - Ensuring all past contributions comply with ECMWF’s [Copyright and Licensing](../Legal/Copyright-And-Licensing.md). - Ensuring the open sourcing guidelines in [Open Sourcing Software at ECMWF](../Legal/Open-Sourcing-Software.md) are followed. -3. **Fork** — ECMWF forks the repository to maintain its own variant while the original remains active. - - Create an empty `default` branch containing a notice linking to the original project. +3. **Fork** — ECMWF forks the repository to maintain its own variant while the original remains active. Follow the standard procedure and notice template in [Forking other repositories](../Contributing%20Upstream/Forking.md). + - Create an empty `default` branch containing a notice linking to the original project (use the notice template in [Forking](../Contributing%20Upstream/Forking.md#readme-template)). - Make `default` the default branch so that it shows as the main entrypoint to the repository. - Disable all GitHub Actions to prevent execution of unverified workflows. - Where applicable, ensure the forked repository complies with ECMWF’s [Copyright and Licensing](../Legal/Copyright-And-Licensing.md). diff --git a/Guidelines/Observability.md b/Guidelines/Observability.md index 0eeb1e6..fcafb09 100644 --- a/Guidelines/Observability.md +++ b/Guidelines/Observability.md @@ -75,7 +75,7 @@ The keywords `MUST`, `SHOULD`, and `MAY` are used as requirement levels: - `MUST`: mandatory requirement for compliance. - `SHOULD`: recommended default; deviations require justification. -- `MAY`: optional behavior. +- `MAY`: optional behaviour. ## 3. Platform Context @@ -182,7 +182,7 @@ Ownership model: ### 3.4 Telemetry Outage and Recovery -Observability design MUST include degraded-mode behavior for periods when +Observability design MUST include degraded-mode behaviour for periods when central telemetry ingestion is unavailable. Minimum requirements: @@ -206,7 +206,7 @@ Ownership model: | Ensure service emits local telemetry in degraded mode | MUST | SHOULD provide guidance | SHOULD validate in production | | Provide buffering/retry/backfill capability in pipeline | N/A | SHOULD | MUST validate operational readiness | | Detect and report ingestion coverage gaps | SHOULD emit health signals | MUST provide platform-level detection | MUST monitor and escalate | -| Maintain outage and recovery runbook | SHOULD contribute service behavior | SHOULD contribute platform behavior | MUST own incident operation | +| Maintain outage and recovery runbook | SHOULD contribute service behaviour | SHOULD contribute platform behaviour | MUST own incident operation | ## 4. Logging Standard @@ -580,8 +580,8 @@ Target-state requirement: Metrics MUST be exposed in Prometheus/OpenMetrics-compatible format. ECMWF services MUST use Prometheus metric types and naming conventions, and MUST expose metrics in a Prometheus/OpenMetrics-compatible text format. -Metrics defined in this section are the source for alerting rules defined in -the Alerting section. +Metrics defined in this section are the source for alerting rules. A dedicated +alerting specification is deferred to a later revision of this document. ### 5.1 Scope and Standard @@ -660,7 +660,7 @@ series, increasing storage/query cost and weakening dashboard/alert signal. - `user_id` - Raw URLs with path parameters - UUIDs or timestamps -- Label values SHOULD be normalized: +- Label values SHOULD be normalised: - Prefer route templates (for example `/api/v1/items/{id}`) over raw paths. - Prefer status classes (`2xx`, `4xx`, `5xx`) when detail is not required. @@ -725,7 +725,7 @@ Before release, teams SHOULD verify: - Metric names, units, and suffixes are compliant. - Required baseline metrics are present. -- Label keys and values are bounded and normalized. +- Label keys and values are bounded and normalised. - No high-cardinality identifiers are emitted as labels. - Histogram buckets are defined and justified. diff --git a/Guidelines/Pull-Requests.md b/Guidelines/Pull-Requests.md index 0c65487..88dad02 100644 --- a/Guidelines/Pull-Requests.md +++ b/Guidelines/Pull-Requests.md @@ -1,11 +1,12 @@ # Pull Requests -Changes to our repositories primary branches, i.e. 'main' / 'master' / +Changes to our repositories' primary branches, i.e. 'main' / 'master' / 'develop' are to be approved through a peer review process based on Pull -Requests (PR). Reviewer and Reviewee have specific responsibilities. This +Requests (PRs). Reviewer and Reviewee have specific responsibilities. This document aims to help both roles to understand their responsibilities. Please -also refer to our branching policy in -[ADR-001](../ADR/ADR-001-Git-Branching-Model.md) +also refer to our branching procedure in +[Branching](./Branching.md) (with the decision and rationale in +[ADR-001](../ADR/ADR-001-Git-Branching-Model.md)) ## Preparation @@ -65,7 +66,7 @@ State disagreements openly so that they can be resolved. Once your reviewers approve, take some time to clean up your PR. Please do not merge commits such as "WIP" or "Fix Review Comment", commits like this make it more difficult in the future to reason about changes. Be aware that this means -rewriting your PRs history. While history rewriting is strongly discourraged +rewriting your PR's history. While history rewriting is strongly discouraged for the primary branches, it is explicitly encouraged on your development branch to create meaningful commits. diff --git a/Guidelines/README.md b/Guidelines/README.md index 00da15c..0afa3c2 100644 --- a/Guidelines/README.md +++ b/Guidelines/README.md @@ -8,7 +8,7 @@ Development guidelines for ECMWF software projects. - [Pull Request Guidelines](./Pull-Requests.md) — responsibilities for reviewers and reviewees, and the PR workflow - [AI Contributions to Software](./Ai-Contributions-To-Software.md) — policy guidelines for AI-assisted code generation and AI agents - [Containerisation](./Containerisation.md) — guidelines for containerising ECMWF software -- [Observability](./Observability.md) — observability guidelines for ECMWF services +- [Observability](./Observability.md) — observability guidelines for ECMWF services *(some sections deferred to a later revision)* - [External Contributions](./External-Contributions.md) — guidelines for accepting external contributions to ECMWF projects - [Security Vulnerability Disclosure](./Security-Vulnerability-Disclosure.md) — coordinated disclosure workflow using GitHub Private Vulnerability Reporting (PVR) and security advisories -- [Testing](./Testing.md) — guidelines for unit, integration and regression testing, and for test suites for services +- [Testing](./Testing.md) — **draft / work in progress**: currently only covers test-data hosting; unit, integration and regression testing standards are still to be written (see `TODO.md`) diff --git a/LICENSE b/LICENSE index e69de29..af25a62 100644 --- a/LICENSE +++ b/LICENSE @@ -0,0 +1,79 @@ +Apache License +Version 2.0, January 2004 +http://www.apache.org/licenses/ + +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + +1. Definitions. + +"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. + +"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. + +"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. + +"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. + +"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. + +"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. + +"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). + +"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. + +"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." + +"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. + +2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. + +3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. + +4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: + + (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. + + You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. + +5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. + +6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. + +7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. + +8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. + +9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. + +END OF TERMS AND CONDITIONS + +APPENDIX: How to apply the Apache License to your work. + +To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. + +Copyright [yyyy] [name of copyright owner] + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +------------------------------------------------------------------------------- + +In applying this licence, ECMWF does not waive the privileges and immunities +granted to it by virtue of its status as an intergovernmental organisation +nor does it submit to any jurisdiction. diff --git a/Languages/Python-Wheels.md b/Languages/Python-Wheels.md index 1ae2482..3a69efc 100644 --- a/Languages/Python-Wheels.md +++ b/Languages/Python-Wheels.md @@ -110,7 +110,7 @@ We don't have any process for deleting broken wheels from PyPI. ### How do I bring external compiled dependencies? Say, for example, you need `libcurl` or `libaec` somewhere in your Compiled library. Ideally, you actually don't. If you do, check whether said library is already available somewhere (as is the case for libcurl and libaec), and consider interfacing with that library through that -- e.g., `eckit` brings in `curl`, so you may want to have `eckit` expose a `curl` call to simplify matters. -If the library is not yet present anywhere, follow the `pre-compile.sh` and `post-build.sh` examples in `eckit/python_wrapper`/`eccodes/python_wrapper` -- those are optional customizable hooks checked for by the repeatable github action, which download and build those dependencies. Note that this process is error-prone and manual, so think twice and test thrice. +If the library is not yet present anywhere, follow the `pre-compile.sh` and `post-build.sh` examples in `eckit/python_wrapper`/`eccodes/python_wrapper` -- those are optional customisable hooks checked for by the repeatable github action, which download and build those dependencies. Note that this process is error-prone and manual, so think twice and test thrice. We are currently in the middle of incorporating https://github.com/ecmwf/cxx-dependencies to the wheel building stack -- ideally include your library there. However, that will still leave in place the need to actually bundle that library somewhere into our stack, which has its own licensing and sizing problems. diff --git a/Languages/README.md b/Languages/README.md index 1e11f32..63542f1 100644 --- a/Languages/README.md +++ b/Languages/README.md @@ -4,6 +4,6 @@ Language-specific guidelines and tooling for ECMWF software projects. ## Contents -- [C++](./C++/README.md) — coding standards, clang-format and clang-tidy configuration +- [C++](./C++/README.md) — C++ tooling *(work in progress: currently only the `.git-blame-ignore-revs` convention; coding standards and clang-format/clang-tidy configuration are still to be added)* - [Python Wheels](./Python-Wheels.md) — building and using Python wheels with compiled libraries -- [Versioning](./Versioning.md) — versioning guidance for multi-language repositories +- [Versioning](./Versioning.md) — **placeholder / to be written**: versioning guidance for multi-language repositories (see `TODO.md`) diff --git a/Legal/Apache-Licence b/Legal/Apache-Licence index 250ddba..af25a62 100644 --- a/Legal/Apache-Licence +++ b/Legal/Apache-Licence @@ -1,190 +1,79 @@ - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - Copyright 1996- European Centre for Medium-Range Weather Forecasts (ECMWF) - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. +Apache License +Version 2.0, January 2004 +http://www.apache.org/licenses/ + +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + +1. Definitions. + +"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. + +"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. + +"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. + +"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. + +"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. + +"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. + +"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). + +"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. + +"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." + +"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. + +2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. + +3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. + +4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: + + (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. + + You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. + +5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. + +6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. + +7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. + +8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. + +9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. + +END OF TERMS AND CONDITIONS + +APPENDIX: How to apply the Apache License to your work. + +To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. + +Copyright [yyyy] [name of copyright owner] + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +------------------------------------------------------------------------------- + +In applying this licence, ECMWF does not waive the privileges and immunities +granted to it by virtue of its status as an intergovernmental organisation +nor does it submit to any jurisdiction. diff --git a/Legal/Copyright-And-Licensing.md b/Legal/Copyright-And-Licensing.md index 2b0a61e..81c76f7 100644 --- a/Legal/Copyright-And-Licensing.md +++ b/Legal/Copyright-And-Licensing.md @@ -2,28 +2,23 @@ This document describes ECMWF's approach to copyright and licensing of software packages. -## Applying the Apache License - ## Context In December 2011, it was decided that ECMWF software packages, when open sourced, will be in future released under the Apache License to legally clarify and simplify the distribution of our software to external users. ## Steps to apply the Apache License -1. If you not familiar with the Apache license, for background information please read [Applying the Apache License](http://www.apache.org/dev/apply-license.html) +1. If you are not familiar with the Apache License, for background information please read [Applying the Apache License](http://www.apache.org/dev/apply-license.html) -2. Each maintainer of a package must include once copy of the full license text by adding the `LICENSE` (note US spelling) file to the root of your repository. You can find the text of the Apache License [here](https://www.apache.org/licenses/LICENSE-2.0.txt). However, you need to make one change to the file (line 178) by adding the Centre and the year for the copyright statement. -``` - Copyright 1996- European Centre for Medium-Range Weather Forecasts (ECMWF) -``` +2. Each maintainer of a package must include one copy of the full licence text by adding a `LICENSE` (note US spelling) file to the root of the repository. The ECMWF `LICENSE` is the **unmodified Apache License 2.0 text followed, at its tail, by ECMWF's intergovernmental notice**. Use the ready-made [Apache-Licence](./Apache-Licence) template in this repository (identical to this repository's own root `LICENSE`); do not hand-edit the Apache text itself. The copyright statement is asserted in the `NOTICE` file and in per-file headers (see below), not by editing the Apache text. -Alternatively, you can use the [Apache-Licence](Apache-Licence) file provided in this repository as a template. +3. A `NOTICE` file **must** be included in the same directory as the `LICENSE`. It is **never empty for ECMWF packages**: it carries ECMWF's copyright and the intergovernmental notice, plus any third-party attributions. Use the [`NOTICE` template](./SPDX-REUSE-templates/NOTICE); follow the [Apache guidelines](http://www.apache.org/legal/src-headers.html#notice) for third-party entries. -3. A correct `NOTICE` file **must** be included in the same directory as the `LICENSE` file and list ALL external code contributions (if any). Please follow the instructions the [Apache guidelines](http://www.apache.org/legal/src-headers.html#notice). Typically, the `NOTICE` file will be empty for ECMWF packages unless you have included third-party code. + The copyright **holder** is normally `European Centre for Medium-Range Weather Forecasts (ECMWF)`. For work funded under an EU programme the holder is the **European Union** (the correct term — not "European Commission"). Where code is co-developed with a partner, assert each holder (e.g. add a `Crown Copyright, Met Office` line for Met Office co-development); see [SPDX and REUSE](./SPDX-and-REUSE.md) for the per-file form. -4. Each original source document (code and documentation, but excluding generated files) **must** include a short license header at the top. To facilitate this, please you may use the script provided in the [Available Tooling](#available-tooling) section below. +4. Each original source document (code and documentation, but excluding generated files) **must** include a short licence header at the top. To facilitate this, you may use the script provided in the [Available Tooling](#available-tooling) section below. -Each source file shall begin with the following license and liability disclaimer: +Each source file shall begin with the following licence and liability disclaimer: ``` (C) Copyright - ECMWF and individual contributors. @@ -48,7 +43,9 @@ does it submit to any jurisdiction. ## Contributors Each repository shall maintain a `CONTRIBUTORS` file in the root of the repository -which lists all contributors. +which lists all contributors. (This is distinct from a `CONTRIBUTING.md`, which is +the optional *contribution guide*; the audit skills treat a missing +`CONTRIBUTING.md` as advisory, but the `CONTRIBUTORS` list is expected here.) Currently we do not have tooling available to generate the contributors list, but this can be created with a bit of manual intervention with git. @@ -63,7 +60,15 @@ git shortlog -s -e --no-merges ### License Header -In case you already have source files that are missing above text -[ECBuild](https://github.com/ecmwf/ecbuild/) provides the +For the **SPDX + REUSE** header (the standard for new files), apply and verify +headers in bulk with the `reuse` tool (see [SPDX and REUSE](./SPDX-and-REUSE.md)): + +``` +reuse annotate --copyright "European Centre for Medium-Range Weather Forecasts (ECMWF)" --license Apache-2.0 --merge-copyrights --recursive . +reuse lint +``` + +For the legacy prose header, [ECBuild](https://github.com/ecmwf/ecbuild/) +provides the [apply\_license.sh](https://github.com/ecmwf/ecbuild/blob/develop/tools/apply_license.sh) -tool to add the above disclaimer to your source files automatically. +tool to add the disclaimer to source files that are missing it. diff --git a/Legal/Open-Sourcing-Software.md b/Legal/Open-Sourcing-Software.md index 56736a2..9eb67f0 100644 --- a/Legal/Open-Sourcing-Software.md +++ b/Legal/Open-Sourcing-Software.md @@ -59,7 +59,7 @@ When open sourcing an ECMWF software, either as an ECMWF member or as a sub-cont [ADR-009 Repository Audit Store](../ADR/ADR-009-Repository-Audit-Store.md). - NB: "licence" is the correct spelling for the noun; "license" for the verb (so "licensing") -- If any external contributions are expected, ensure the contribution licence agreement plugin (CLA Assistant) to the repository. - - Contact GitHub admin for this - typically is done by ensuring your PR's use a template that presents the CLA +- If any external contributions are expected, add the Contributor Licence Agreement plugin (CLA Assistant) to the repository (see the [Contributor Licence Agreement](./Contributor-License-Agreement.md)). + - Contact a GitHub admin for this — typically done by ensuring the repository provides a PR template that presents the CLA - Ideally some automatic CI tests should be set up, that any pull request can be checked. Contact Development section to arrange for integration into ECMWF CI/CD system. - Make sure to stress that any contribution should come with documentation and tests! diff --git a/Legal/README.md b/Legal/README.md index 0b23872..1a2a6f2 100644 --- a/Legal/README.md +++ b/Legal/README.md @@ -9,4 +9,4 @@ Legal guidance for ECMWF software projects covering licensing, copyright, and re - [Open Sourcing Software](./Open-Sourcing-Software.md) — checklist and guidelines for open sourcing ECMWF software - [Contributor Licence Agreement](./Contributor-License-Agreement.md) — CLA for contributions to ECMWF software - [Requesting a New Repository](./Requesting-New-Repository.md) — process for requesting or renaming GitHub repositories -- [Apache Licence](./Apache-Licence) — full text of the Apache License 2.0 +- [Apache Licence](./Apache-Licence) — the ECMWF top-level `LICENSE` template: the Apache License 2.0 text with ECMWF's intergovernmental notice appended at the tail diff --git a/Legal/Requesting-New-Repository.md b/Legal/Requesting-New-Repository.md index 960bbc0..a65f219 100644 --- a/Legal/Requesting-New-Repository.md +++ b/Legal/Requesting-New-Repository.md @@ -50,4 +50,6 @@ Requesting a **Public** repository (or changing an existing repository's visibil If your repository requires public visibility, you must complete the official open-sourcing approval procedure. Your repository will be first made Private and only made Public once the open-sourcing process is complete and approved. -Please review and follow the guidelines documented here: [Open Sourcing Software Procedure](https://github.com/ecmwf/codex/blob/main/Legal/Open-Sourcing-Software.md) +Please review and follow the guidelines documented here: [Open Sourcing Software Procedure](./Open-Sourcing-Software.md) + +New repositories should also follow the [Repository Structure](../Repository%20Structure/README.md) conventions and, for public or eventually-public repositories, ensure the Contributor Licence Agreement is in place (see [External Contributions](../Guidelines/External-Contributions.md) and the [Contributor Licence Agreement](./Contributor-License-Agreement.md)). diff --git a/MARS language/MARS-001-MARS-Model-Keyword-for-DestinE-ODEDT.md b/MARS language/MARS-001-MARS-Model-Keyword-for-DestinE-ODEDT.md index ff7d182..7adc026 100644 --- a/MARS language/MARS-001-MARS-Model-Keyword-for-DestinE-ODEDT.md +++ b/MARS language/MARS-001-MARS-Model-Keyword-for-DestinE-ODEDT.md @@ -1,10 +1,12 @@ # MARS Language Decision Record 001: MARS Model Keyword for DestinE ODEDT (DE330) -[~~Proposed~~ | **Accepted** | ~~Deprecated~~ | ~~Superseded by [ADR-XXX]~~] +[~~Proposed~~ | **Accepted** | ~~Deprecated~~ | ~~Superseded by [ADR-XXX]~~] + +> Originally raised as [ADR-006](../ADR/ADR-006-MARS-Model-Keyword-for-DestinE-ODEDT.md), which was withdrawn and moved here; this record is the authoritative decision. # Last Updated -10.03.2026 +2026-03-10 # Context diff --git a/MARS language/MARS-002-flightLevels.md b/MARS language/MARS-002-flightLevels.md index 57d2cd6..b42f79e 100644 --- a/MARS language/MARS-002-flightLevels.md +++ b/MARS language/MARS-002-flightLevels.md @@ -7,7 +7,7 @@ 2026-05-19 ## Context -Flight levels are a standardized way of expressing an aircraft’s altitude, primarily used during cruise flight at higher altitudes. Instead of referencing height above sea level directly, flight levels are based on the International Standard Atmosphere (ISA), which assumes a mean sea level pressure of 1013.25 hPa, a temperature of 15 °C at sea level, etc. This ensures that all aircraft in the same airspace use a common reference, maintaining safe vertical separation regardless of local pressure variations. Flight levels are expressed in hundreds of feet, so Flight Level 350 (FL350) corresponds to approximately 35,000 feet. +Flight levels are a standardised way of expressing an aircraft’s altitude, primarily used during cruise flight at higher altitudes. Instead of referencing height above sea level directly, flight levels are based on the International Standard Atmosphere (ISA), which assumes a mean sea level pressure of 1013.25 hPa, a temperature of 15 °C at sea level, etc. This ensures that all aircraft in the same airspace use a common reference, maintaining safe vertical separation regardless of local pressure variations. Flight levels are expressed in hundreds of feet, so Flight Level 350 (FL350) corresponds to approximately 35,000 feet. In ecCharts, Clear-Air-Turbulence (CAT) is provided on a set of different flight levels, and this data should be produced in GRIB2 in 50r1. [See an example of CAT on flight levels in ecCharts](https://charts.ecmwf.int/products/medium-cat?flight_level=9144&projection=opencharts_europe) @@ -19,11 +19,11 @@ There is no existing mechanism to encode flight levels directly in GRIB2. ### Options Considered 1. Pressure levels: The data could be encoded using pressure levels following the barometric formula conversion. For instance, FL340 would be encoded using typeOfLevel=pressure with a combined scale value and scaled factor equal to 25000Pa. However, archiving them under levtype=pl is not very good, as this would introduce new pressure levels with unusual values for a standard user. These new pressure levels would contain only a single parameter, which is Clear-Air-Turbulence creating a fair amount of unused entry in the MARS layout and introducing invalid combination of parameter/levelist in the MARS catalogue. This is also a very specific application so it is probably good to have the parameter(s) for aviation purposes on a separate levtype so that pl contains parameters for a wider user group. This can be achieved in ecCodes using the same mechanism which put 2m and 10m parameters on levtype=sfc. -2. Height levels would be another option, but flight levels are not a ‘real’ height as this relates to the assumption of having the standardized atmospheric conditions. For instance, FL350 would be encoded using typeOfLevel=height with a combined scale value and scaled factor equal to 10668m (35,000 feet). All other parameters are referenced to the earth’s surface. This is also a very specific application why it is probably good to have the parameter(s) for aviation purposes on a separate levtype so that hl contains parameters for a wider user group. As for the pressure level option, this can also be achieved in ecCodes using the remapping mechanism. +2. Height levels would be another option, but flight levels are not a ‘real’ height as this relates to the assumption of having the standardised atmospheric conditions. For instance, FL350 would be encoded using typeOfLevel=height with a combined scale value and scaled factor equal to 10668m (35,000 feet). All other parameters are referenced to the earth’s surface. This is also a very specific application why it is probably good to have the parameter(s) for aviation purposes on a separate levtype so that hl contains parameters for a wider user group. As for the pressure level option, this can also be achieved in ecCodes using the remapping mechanism. 3. Creating a new typeOfLevel “Flight levels” is the third possible option. For now, we would use a local entry in the GRIB2 message and encode the flight level in the GRIB record. FL340 would get typeOfFirstFixedSurface=flight level and level=340 instead of pressure level and 25,000Pa. In parallel, we would propose a new code table entry in the coming WMO Fast-track for flight levels. This means that with the availability of an official code table entry, the only change in the meta-data of the data would be just a change of the code number in typeOfFirstFixedSurface. In terms of high-level keys, there would be no change in behaviour. For the MARS language, we propose to map the new TypeOfLevel to a new levtype value “fl” (flight level) which would work together with the corresponding flight level value encoded in the MARS levelist keyword. ### Analysis -We considered encoding the flight levels as pressure levels mapped to levtype=fl but we noted the possible clashes with existing pressure levels already produced under levtype=pl. This is indeed the case for FL340 clashing with PL250. We also considered encoding the flight levels as height above ground levels mapped to levtype=fl. With height levels, the risk for clashes is virtually non-existant because the lowest flight level FL100 corresponds to a height of 3048m. However, flight levels are defined with respect to an abstract reference assuming a standard atmosphere, while height levels are referenced to the Earth’s surface, resulting in a discrepancy between them. The third option, i.e. creating a new type of level does not lead to potential clashes with existing data but it requires proposing them to WMO with an official solution only available in November 2026. +We considered encoding the flight levels as pressure levels mapped to levtype=fl but we noted the possible clashes with existing pressure levels already produced under levtype=pl. This is indeed the case for FL340 clashing with PL250. We also considered encoding the flight levels as height above ground levels mapped to levtype=fl. With height levels, the risk for clashes is virtually non-existent because the lowest flight level FL100 corresponds to a height of 3048m. However, flight levels are defined with respect to an abstract reference assuming a standard atmosphere, while height levels are referenced to the Earth’s surface, resulting in a discrepancy between them. The third option, i.e. creating a new type of level does not lead to potential clashes with existing data but it requires proposing them to WMO with an official solution only available in November 2026. Considering all options, we recommend option 3, i.e. to archive the data under newly created flight levels. This allows to add more parameters for aviation meteorology in the future which are currently also produced under the standard pressure levels under the flight levels. An encoding example: ```bash diff --git a/MARS language/MARS-003-model-errors-with-coeffs.md b/MARS language/MARS-003-model-errors-with-coeffs.md index 77c5c9b..fd721f2 100644 --- a/MARS language/MARS-003-model-errors-with-coeffs.md +++ b/MARS language/MARS-003-model-errors-with-coeffs.md @@ -1,7 +1,7 @@ # MARS Language Decision Record 003: Model Error Coefficients Index ## Status -[Proposed | **Accepted** | Deprecated | Superseded by [ADR-XXX]] +[~~Proposed~~ | **Accepted** | ~~Deprecated~~ | ~~Superseded by [ADR-XXX]~~] ## Last Updated 2026-01-09 diff --git a/MARS language/MARS-004-land-data-assimilation-system.md b/MARS language/MARS-004-land-data-assimilation-system.md index 49c5d84..ce41884 100644 --- a/MARS language/MARS-004-land-data-assimilation-system.md +++ b/MARS language/MARS-004-land-data-assimilation-system.md @@ -18,7 +18,7 @@ which need to be distinguished. Data are needed in the FDB and will be archived Data from LDAS include analysis values and forecast values (e.g. surface fluxes). Analysis and forecast values for a given valid time will exist for the BRT and for the NRT at multiple lead times. The BRT will typically run each day covering the period up to 3 days previous. The NRT -will run daily with a 5-day spinup and will be intialized from the BRT product. Additionally, +will run daily with a 5-day spinup and will be initialised from the BRT product. Additionally, diagnostic statistics such as monthly means will be produced at the end of each calendar month. Operational configurations will run in parallel for each of MR, S2S and SEAS. The BRT component @@ -63,15 +63,15 @@ We considered introducing a new class `ld` for ldas operational data, but this w alternative solution for research data. This would either complicate life for everyone by structuring data differently, or require a complex separation of both research and operational data within `class=ld`. By contrast, introducing new streams for the instantaneous and derived `ldas` / `ldst` -statistis allows us to have the same layout under `class=od` and `rd`, and even for member state +statistics allows us to have the same layout under `class=od` and `rd`, and even for member state data if required in the future. It avoids relying on specific `expver` ranges reserved for -research experiments, and facilitates a clean introduciton of the required MARS keywords under +research experiments, and facilitates a clean introduction of the required MARS keywords under the new streams. -The same challenges of distinguishing between different model configurations exists in this case +The same challenges of distinguishing between different model configurations exist in this case as for that of hydrological data (see MARS-005), and it makes sense to introduce and use the same keyword and language structure, `configuration`, for this purpose. The structure of the -confguration keywourd should be similarly dash-delineated. Using the `configuration` keyword in +configuration keyword should be similarly dash-delimited. Using the `configuration` keyword in the same way will enhance the consistency of MARS. It was considered that using the names of the streams (`enfo/eefo/mmsf`) in the `configuration` @@ -117,7 +117,7 @@ and daily statistics, and using the `time` keyword also enables monthly synoptic ### Related Decisions - - [MARS-005 CEMS C3S Hydrology](MARS-005-CEMS-C3S-Hydrology.md) - This decisision is in + - [MARS-005 CEMS C3S Hydrology](MARS-005-CEMS-C3S-Hydrology.md) - This decision is in line with the implementation for hydrological data. ## Consequences diff --git a/MARS language/MARS-005-CEMS-C3S-Hydrology.md b/MARS language/MARS-005-CEMS-C3S-Hydrology.md index e0f60a9..34c319d 100644 --- a/MARS language/MARS-005-CEMS-C3S-Hydrology.md +++ b/MARS language/MARS-005-CEMS-C3S-Hydrology.md @@ -38,9 +38,9 @@ issue is observed between the COSMO and ICON models at DWD. Sufficient data is p to identify this data further. As such we propose a new MARS keyword `forcing` which will combine the originating centre and the associated -atmospheric model. The values will be construted of the form `-`. +atmospheric model. The values will be constructed of the form `-`. -Finally, we propose restructuring the overall archive. Currently in CEMS th edifferent hydrological versions for +Finally, we propose restructuring the overall archive. Currently in CEMS the different hydrological versions for the European and the global domains are archived into different `stream`s under `class` `ce`. We introduce instead four distinct `class`es for the European CEMS / C3S and their global counterparts. diff --git a/MARS language/MARS-006-Land-Surface-Tile-Data.md b/MARS language/MARS-006-Land-Surface-Tile-Data.md index edea99a..ec59044 100644 --- a/MARS language/MARS-006-Land-Surface-Tile-Data.md +++ b/MARS language/MARS-006-Land-Surface-Tile-Data.md @@ -4,7 +4,7 @@ # Last Updated -19.03.2026 +2026-03-19 # Context diff --git a/MARS language/MARS-007-Timespan-Absence.md b/MARS language/MARS-007-Timespan-Absence.md index cc98c49..38c657d 100644 --- a/MARS language/MARS-007-Timespan-Absence.md +++ b/MARS language/MARS-007-Timespan-Absence.md @@ -1,4 +1,4 @@ -# MARS Languge Decision Record 007: Handling (absence of) timespan in post-MTG2 data +# MARS Language Decision Record 007: Handling (absence of) timespan in post-MTG2 data [~~Proposed~~ | **Accepted** | ~~Deprecated~~ | ~~Superseded by [ADR-XXX]~~] @@ -194,10 +194,9 @@ Configuration changes required * pre-MTG2 data will be archived with `timespan` absent * post-MTG2 data will be archived with `timespan=none` * Listing will return values as indexed (i.e. depending on pre/post-MTG2) - * Retrieval will work for both `timespan=none` and `timespan` absent as the language semantic is updated to treat these two values the same.. + * Retrieval will work for both `timespan=none` and `timespan` absent as the language semantics are updated to treat these two values the same. * In MARS Server * We select on the presence of `timespan` *in an archive request* to select the tree structure. - * * We need to use `PSimpleNodeDefault` to select the correct branch during retrieve requests if `timespan` is not specified in the request. * In verification, we can still verify that an archive request matches *exactly* to the contents of the MARS namespace (we can run an identical check to that in the client). diff --git a/MARS language/MARS-Template.md b/MARS language/MARS-Template.md index 0b0f267..87e6c7b 100644 --- a/MARS language/MARS-Template.md +++ b/MARS language/MARS-Template.md @@ -1,4 +1,4 @@ -# MARS Languge Decision Record XXX: +# MARS Language Decision Record XXX: [**Proposed** | ~~Accepted~~ | ~~Deprecated~~ | ~~Superseded by [ADR-XXX]~~] diff --git a/Principles/Code-Of-Conduct.md b/Principles/Code-Of-Conduct.md index cd06a5d..79b52bb 100644 --- a/Principles/Code-Of-Conduct.md +++ b/Principles/Code-Of-Conduct.md @@ -2,7 +2,7 @@ ## Our Pledge -In developing software and services at ECMWF and with our community of Member and Cooperating States, we are +In developing software and services at ECMWF and with our community of Member and Co-operating States, we are committed to fostering an inclusive, respectful, and harassment-free community in line with [ECMWF's Open Source Principles](./Open-Source-Principles.md), which are inspired by the [United Nations Open Source Principles](https://opensource.un.org/en/news/united-nations-open-source-principles). diff --git a/Principles/Open-Source-Principles.md b/Principles/Open-Source-Principles.md index 055595c..6448fb2 100644 --- a/Principles/Open-Source-Principles.md +++ b/Principles/Open-Source-Principles.md @@ -24,13 +24,13 @@ ECMWF does not only publish its own software as open source — it also seeks to *Making security a priority in all software projects.* -Software developed at ECMWF is deployed in operational forecasting chains, national meteorological services, and critical infrastructure. Security must therefore be considered from the outset, not bolted on later. This includes responsible management of dependencies, timely response to vulnerability disclosures (see the [Security Vulnerability Disclosure](../Guidelines/Security-Vulnerability-Disclosure.md) procedure), use of automated security scanning in CI pipelines, and following best practices for secrets management and access control. +Software developed at ECMWF is deployed in operational forecasting chains, national meteorological services, and critical infrastructure. Security must therefore be considered from the outset, not bolted on later. This includes responsible management of dependencies, timely response to vulnerability disclosures (see the [Security Vulnerability Disclosure](../Guidelines/Security-Vulnerability-Disclosure.md) procedure), running automated security checks — such as dependency-vulnerability and static-analysis scanning — where practical, and following best practices for secrets management and access control. ## 4 — Foster Inclusive Participation *Enabling and facilitating diverse and inclusive contributions.* -ECMWF's open source projects serve a global community spanning meteorological services, academic research institutions, and the private commercial sector. We welcome contributions regardless of organisational affiliation, geographic location, or background. This means maintaining clear contribution guidelines (see [External Contributions](../External%20Contributions)), responding constructively and politely to issues and pull requests. Inclusive participation strengthens the software and broadens the community that sustains it. +ECMWF's open source projects serve a global community spanning meteorological services, academic research institutions, and the private commercial sector. We welcome contributions regardless of organisational affiliation, geographic location, or background. This means maintaining clear contribution guidelines (see [External Contributions](../Guidelines/External-Contributions.md)), and responding constructively and politely to issues and pull requests. Inclusive participation strengthens the software and broadens the community that sustains it. ## 5 — Design for Reusability diff --git a/Project Maturity/README.md b/Project Maturity/README.md index 6cb5c94..7f0914f 100644 --- a/Project Maturity/README.md +++ b/Project Maturity/README.md @@ -1,70 +1,44 @@ -# Maturity Levels of Software +# Project Maturity Software maturity refers to the stages a software project goes through during its lifecycle. While ECMWF runs operational services, this does not necessarily mean that the underlying software is operationally supported for external users. The following maturity levels provide an indication of how well-supported and operationally ready ECMWF software is. However, as with any open-source software, users should evaluate its suitability for their needs and use it at their own risk. -Repositories should show a badge indicating their software maturity level, and the following disclaimer, adjusted to the correct level: +Every repository should show, near the top of its README (after the title and an optional short synopsis), a badge indicating its maturity level and the matching disclaimer: -> \[!IMPORTANT\] -> This software is **Graduated** and subject to ECMWF's guidelines on [Software Maturity](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity). +```markdown +[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#) -# Graduated +> [!IMPORTANT] +> This software is **** and subject to ECMWF's guidelines on [Software Maturity](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity). +``` -[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/graduated_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#graduated) - -Graduated projects are ready for operations, and typically used by ECMWF and/or its Member and Cooperating states in operations. This does not mean ECMWF gives operational support to the software itself. +Replace ``/`` with the level below (e.g. `graduated` / `Graduated`). - [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/graduated_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#graduated)) +## Graduated - > \[!IMPORTANT\] - > This software is **Graduated** and subject to ECMWF's guidelines on [Software Maturity](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity). +[![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/graduated_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#graduated) +Graduated projects are ready for operations, and typically used by ECMWF and/or its Member and Co-operating States in operations. This does not mean ECMWF gives operational support to the software itself. -# Incubating +## Incubating [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/incubating_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#incubating) -Incubating projects are mostly feature-complete and the interface is mostly stable. If this software is to be used in operational systems you are **strongly advised to use a released tag in your system configuration**, and you should be willing to accept incoming changes and bug fixes that require adaptations on your part. ECMWF **may be using** this software in operations and abides by the same caveats. - - [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/incubating_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#incubating) +Incubating projects are mostly feature-complete and the interface is mostly stable. If this software is to be used in operational systems you are **strongly advised to use a released tag in your system configuration**, and you should be willing to accept incoming changes and bug fixes that require adaptations on your part. ECMWF **may be using** this software in operations and abides by the same caveats. - - > \[!IMPORTANT\] - > This software is **Incubating** and subject to ECMWF's guidelines on [Software Maturity](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity). - -# Emerging +## Emerging [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/emerging_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#emerging) Emerging projects are in the early stages of development. There is a clear project goal, but they are not yet feature-complete and not stable. - [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/emerging_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#emerging) - - > \[!IMPORTANT\] - > This software is **Emerging** and subject to ECMWF's guidelines on [Software Maturity](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity). - - -# Sandbox +## Sandbox [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/sandbox_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#sandbox) Sandbox projects are experimental, proof-of-concept. Expect frequent changes, incomplete features and instability. - [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/sandbox_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#sandbox) - - > \[!IMPORTANT\] - > This software is **Sandbox** and subject to ECMWF's guidelines on [Software Maturity](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity). - - -# Archived +## Archived [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/archived_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#archived) -Projects that have reached the end of their lifecycle. -They are no longer actively maintained or developed. -These projects may still be available for reference or historical purposes, but they should not be used for any active development or operational purposes. - - [![Static Badge](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity/archived_badge.svg)](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity#archived) - - > \[!IMPORTANT\] - > This software is **Archived** and subject to ECMWF's guidelines on [Software Maturity](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity). - +Projects that have reached the end of their lifecycle. They are no longer actively maintained or developed. These projects may still be available for reference or historical purposes, but they should not be used for any active development or operational purposes. diff --git a/README.md b/README.md index 3113246..3ade839 100644 --- a/README.md +++ b/README.md @@ -10,21 +10,16 @@ The Codex is a set of principles and guidelines for development of software and # Contents -- [Code of Conduct](./Principles/Code-Of-Conduct.md) -- [Principles](./Principles) -- [Architectural Decision Records](./ADR) +- [Principles](./Principles) — including the [Code of Conduct](./Principles/Code-Of-Conduct.md) and the [Open Source Principles](./Principles/Open-Source-Principles.md) +- [Architectural Decision Records (ADR)](./ADR) +- [MARS Language Decision Records](./MARS%20language) - [Repository Structure](./Repository%20Structure) - [Project Maturity](./Project%20Maturity) -- [Testing](./Guidelines/Testing.md) -- [Containerisation](./Guidelines/Containerisation.md) -- [Observability](./Guidelines/Observability.md) - [ECMWF Software EnginE (ESEE)](./ESEE) -- [External Contributions](./Guidelines/External-Contributions.md) -- [Security Vulnerability Disclosure](./Guidelines/Security-Vulnerability-Disclosure.md) -- [Documentation and Training](./Documentation%20and%20Training/) -- [Contributing to External Projects](./Contributing%20Upstream/) -- [Guidelines](./Guidelines/) -- [Languages](./Languages/) -- [Legal](./Legal/) -- [Software Management Plan](./Software%20Management%20Plan/) -- [Agent Skills](./Agent%20Skills/) +- [Guidelines](./Guidelines) — branching, pull requests, testing, containerisation, observability, external contributions, AI contributions, and security vulnerability disclosure +- [Languages](./Languages) +- [Legal](./Legal) +- [Documentation and Training](./Documentation%20and%20Training) +- [Contributing to External Projects](./Contributing%20Upstream) +- [Software Management Plan](./Software%20Management%20Plan) +- [Agent Skills](./Agent%20Skills) diff --git a/Repository Structure/Example.md b/Repository Structure/Example.md index 16b06d0..d27a27d 100644 --- a/Repository Structure/Example.md +++ b/Repository Structure/Example.md @@ -3,7 +3,7 @@

ESEE Foundation - + Maturity Sandbox @@ -36,6 +36,9 @@ # foo +> [!IMPORTANT] +> This software is **Sandbox** and subject to ECMWF's guidelines on [Software Maturity](https://github.com/ecmwf/codex/raw/refs/heads/main/Project%20Maturity). + > [!TIP] > Write here the "mission statement" of the project. > It should be a short paragraph that describes the purpose of the project (why have we built this?) and what it does. It should be written in a way that is easy to understand for someone who is not familiar with the project. @@ -69,4 +72,4 @@ python -m pip install foo ## License -© 2025 ECMWF. All rights reserved. \ No newline at end of file +[Apache License 2.0](LICENSE) In applying this licence, ECMWF does not waive the privileges and immunities granted to it by virtue of its status as an intergovernmental organisation nor does it submit to any jurisdiction. \ No newline at end of file diff --git a/Repository Structure/README.md b/Repository Structure/README.md index 9b7af89..24b13d0 100644 --- a/Repository Structure/README.md +++ b/Repository Structure/README.md @@ -34,7 +34,13 @@ my-repo/ ├── bin/ ├── docs/ ├── examples/ # optional -├── LICENSE +├── LICENSE # Apache 2.0 + ECMWF intergovernmental notice +├── LICENSES/ # REUSE: unmodified licence texts (Apache-2.0.txt) +├── NOTICE # ECMWF copyright + intergovernmental notice +├── REUSE.toml # REUSE annotations for non-commentable files +├── SECURITY.md # recommended: vulnerability-disclosure policy +├── CITATION.cff # recommended: citation metadata +├── CONTRIBUTORS # list of contributors ├── pyproject.toml ├── README.md # deviation from the guide ├── setup.py # VERY shallow, just calls setup() or something @@ -47,14 +53,25 @@ my-repo/ └── tox.ini ``` -Note the extra src directory. This prevents setuptool's find_package from adding unwanted code to the installation, without having to add exceptions for tests, examples, etc. It also forces the package to be pip install'ed before running the tests, which helps checking that the package will work as intended once installed. Simple use case: it won't import the local package if you run pytest in the folder. +The licensing artefacts (`LICENSE`, `LICENSES/`, `NOTICE`, `REUSE.toml`, per-file +SPDX headers) follow [Copyright and Licensing](../Legal/Copyright-And-Licensing.md) +and [SPDX and REUSE](../Legal/SPDX-and-REUSE.md); `SECURITY.md` and `CITATION.cff` +are described below. + +Note the extra src directory. This prevents setuptools' find_packages from adding unwanted code to the installation, without having to add exceptions for tests, examples, etc. It also forces the package to be installed with `pip` before running the tests, which helps check that the package will work as intended once installed. Simple use case: it won't import the local package if you run pytest in the folder. ### C/C++ ``` ├── CMakeLists.txt -├── LICENSE +├── LICENSE # Apache 2.0 + ECMWF intergovernmental notice +├── LICENSES/ # REUSE: unmodified licence texts +├── NOTICE # ECMWF copyright + intergovernmental notice +├── REUSE.toml # REUSE annotations for non-commentable files +├── SECURITY.md # recommended +├── CITATION.cff # recommended +├── CONTRIBUTORS ├── README.md ├── VERSION ├── cmake diff --git a/Software Management Plan/README.md b/Software Management Plan/README.md index b865e95..13845d3 100644 --- a/Software Management Plan/README.md +++ b/Software Management Plan/README.md @@ -4,13 +4,13 @@ A software management plan (SMP) should be provided for all projects which contr This is to ensure that contributions are appropriate, visible, follow guidelines and there is a clear roadmap for development and ownership. -A SMP is required for all projects where the lead developers are external contributors +An SMP is required for all projects where the lead developers are external contributors and should be provided in the planning stages of the project. -For internal contributors, a SMP is required to progress a project from the *Sandbox* +For internal contributors, an SMP is required to progress a project from the *Sandbox* to the *Emerging* [Project Maturity](../Project%20Maturity/) level. -Please use the [template SMP](./SMP-template.docx) to create your SMP, it contains much of the description -provide below. +Please use the [template SMP](./SMP-template.docx) to create your SMP; it contains much of the description +provided below. ## 1. Project Overview @@ -26,7 +26,7 @@ Provide a concise overview to allow others to understand the project and its pur ## 2. Alignment With ECMWF Strategy and Architecture -How do the proposed software developments fit into ECMWF’s broader organizational goals? +How do the proposed software developments fit into ECMWF’s broader organisational goals? - **Alignment with the ECMWF software stack** - New components @@ -140,7 +140,7 @@ Include: - **Developer documentation** - What will it cover, e.g. API, architecture, contribution guidelines. - **Documentation location** - - read-the-docs/github-pages/confluence? + - Read the Docs, GitHub Pages, or `sites.ecmwf.int` (see [Documentation and Training](../Documentation%20and%20Training/README.md)) - **Support model** - Who is the point of contact for support - Where and how is support provided, e.g. github issues and/or jira tickets. diff --git a/TODO.md b/TODO.md index f67351f..898856f 100644 --- a/TODO.md +++ b/TODO.md @@ -3,26 +3,36 @@ This document captures known gaps, incomplete sections, and issues identified across the Codex repository. Contributions addressing any of these items are welcome via pull request. +A consistency pass has fixed the cross-document inconsistencies, broken links, +licensing contradictions, ADR status/formatting drift, and spelling/house-style +issues. The items below are the remaining **content gaps** — sections that are +advertised but not yet written. Their index entries now flag them as +work-in-progress. + --- ## High Priority -- [ ] **Guidelines/Testing.md** — File opens with an explicit `TODO`. No actual testing guidelines exist beyond test data hosting via `sites.ecmwf.int`. Needs unit, integration, and regression testing standards. -- [ ] **Languages/Versioning.md** — Entire file is a single-line placeholder: *"How do we manage multi-language repositories with a single version number?"*. Needs a versioning policy. -- [ ] **LICENSE** (root) — File is empty. The licence text exists in `Legal/Apache-Licence` but the root `LICENSE` file that GitHub uses for badge detection and display contains no content. +- [ ] **Guidelines/Testing.md** — Still only covers test-data hosting. Needs unit, integration, and regression testing standards (the README now marks it as draft; other docs' "tests are mandatory" wording depends on this being written). +- [ ] **Languages/Versioning.md** — Still a single-line placeholder. Needs a versioning policy for multi-language repositories. ## Medium Priority -- [ ] **Languages/C++/** — Only covers `.git-blame-ignore-revs` configuration. The Languages README promises coding standards, clang-format, and clang-tidy configuration, but none of these are present. -- [ ] **Documentation and Training/README.md** — Skeletal (16 lines). No documentation standards, templates, API documentation requirements, or style guides. The Software Management Plan section contains more documentation guidance than this dedicated section. -- [ ] **ESEE/README.md** — Conceptual category descriptions only. Missing component listings, architecture details, and diagrams showing which software packages belong to which ESEE category. -- [ ] **Guidelines/Observability.md** — Explicitly defers several areas to future revisions: full tracing specification, environment-specific collection pipelines, and an alerting section (referenced but not included). +- [ ] **Languages/C++/** — Only the `.git-blame-ignore-revs` convention exists. Needs the promised C++ coding standards and clang-format / clang-tidy configuration. +- [ ] **Documentation and Training/README.md** — Skeletal. Needs documentation standards, templates, API-documentation requirements, and style guides (the SMP §7 documentation guidance should eventually be consolidated here). +- [ ] **ESEE/README.md** — Conceptual category descriptions only. Missing component listings, architecture details, and diagrams showing which packages belong to which ESEE category. +- [ ] **Guidelines/Observability.md** — Defers the full tracing specification, environment-specific collection pipelines, and a dedicated alerting specification to future revisions. ## Low Priority -- [ ] **ADR/ADR-005-Storing-Of-ICON-Grid-Files.md** — Title inside the document reads "ADR 004" but the filename is `ADR-005`. Numbering mismatch needs correcting. -- [ ] **MARS language/MARS-Template.md** — Typo in title: "Languge" should be "Language". -- [ ] **MARS language/MARS-007-Timespan-Absence.md** — Same "Languge" typo in title. Also contains an empty bullet point (`* `) around line 200. -- [ ] **Contributing Upstream/README.md** — Single-line index file. Could benefit from a brief introductory paragraph explaining the purpose of the section. -- [ ] **Repository Structure/Example.md** — Only provides a Python project README template. No equivalent example for a C++ project. -- [ ] **Languages/Python-Wheels.md** — Notes that automated release triggering is pending and pin derivation for Python interface wheels has a known deficiency. +- [ ] **Repository Structure/Example.md** — Provides a Python project README template only. No equivalent example for a C++ project. +- [ ] **Languages/Python-Wheels.md** — Automated release triggering is pending, and pin derivation for Python interface wheels has a known deficiency. + +## Resolved by the consistency pass + +- Root `LICENSE` populated (Apache 2.0 + ECMWF intergovernmental notice). +- Broken links fixed (Principles → External Contributions; ADR-Guidelines → ADR index; Example.md maturity badge). +- Licensing contradictions reconciled (NOTICE "never empty", canonical LICENSE composition, EU/partner copyright holder, "line 178" removed, `Example.md` "All rights reserved" replaced). +- ADR statuses/formatting normalised (ADR-005 title number, ADR-007 status, ADR-002/006 headings, template brackets); ADR-006 reduced to a stub linking MARS-001. +- MARS typos ("Languge"), empty bullet, strikethrough/date-format drift fixed. +- British house style applied (Containerisation, Observability, SMP, ADR-004, ESEE); "Co-operating States" standardised.