diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..8d05175 --- /dev/null +++ b/.gitignore @@ -0,0 +1,10 @@ +data +logs +drafts/ + +# Claude Code +.claude/ +CLAUDE.md + +# Git sync +.git.nosync/ diff --git a/.markdownlint.yml b/.markdownlint.yml new file mode 100644 index 0000000..60d8c1a --- /dev/null +++ b/.markdownlint.yml @@ -0,0 +1,36 @@ +# Markdownlint configuration for GA4GH TASC repository +# Prose-heavy documentation project — disable rules that conflict with long-form writing + +# Disable line length — prose documents naturally exceed 80 chars +MD013: false + +# Allow both dash and asterisk for unordered lists (existing docs use both) +MD004: false + +# Allow flexible list indentation (3-space indent used in ADRs) +MD007: false + +# Allow ordered lists with sequential numbering (1. 2. 3.) not just (1. 1. 1.) +MD029: + style: ordered + +# Allow headings without surrounding blank lines in metadata blocks +MD022: false + +# Allow lists without surrounding blank lines (common in ADR consequence sections) +MD032: false + +# Allow inline HTML (used in some recommendation tables) +MD033: false + +# Allow duplicate headings (e.g. "Parameters" under different parent sections) +MD024: false + +# Allow multiple top-level headings (some docs use bold text parsed as h1) +MD025: false + +# Disable table column style — compact vs padded varies across docs +MD060: false + +# Allow bare URLs in reference sections +MD034: false diff --git a/README.md b/README.md index b949b76..1807acc 100644 --- a/README.md +++ b/README.md @@ -8,11 +8,39 @@ While TASC functions as a central decision-making body, all of its decisions are TASC organizes and monitors its technical alignment activities through a [GitHub Project board](https://github.com/orgs/ga4gh/projects/9/views/1), where you can find and track the progress of current issues and initiatives. +## Repository Structure + +This repository is organized into the following directories: + +``` +/ +├── README.md (overview & quick reference) +├── /adr/ (Architectural Decision Records) +├── /governance/ (TASC operating charter) +├── /recommendations/ (TASC outputs) +├── /drafts/ (work in progress) +└── /service-info/ (service type registry) +``` + +### Directory Descriptions + +- **`/adr/`** - Architectural Decision Records documenting key technical decisions made by TASC. Each ADR captures what was decided, why, and the consequences. ADRs are immutable historical records. See [adr/template.md](adr/template.md) for the standard format. + +- **`/governance/`** - TASC's governance charter defining how TASC operates, including membership, meeting format, and decision-making processes. + +- **`/recommendations/`** - Approved TASC recommendations and policy documents that provide guidance to GA4GH work streams. These include API standards, process guidelines, and technical specifications. + +- **`/drafts/`** - Work-in-progress documents under development. Documents move from drafts to recommendations upon approval. + +- **`/service-info/`** - The GA4GH Service Info Type Registry, which maintains standardized service type identifiers for GA4GH API servers enabling service discovery. + ## Table of Contents + +- [Repository Structure](#repository-structure) - [TASC Structure](#tasc-structure) -- [TASC Members](#tasc-members) +- [TASC Members](#tasc-members) - [Group Meeting Format](#group-meeting-format) -- [Decision-Making Process](#decision-making-process) +- [Decision-Making Process](#decision-making-process) - [Issue labels](#issue-labels) ## TASC Structure @@ -159,7 +187,7 @@ For asynchronous discussion, members use: 6. If approval not reached through multiple rounds, TASC Leadership makes final decision 7. Leadership can take decisions if two consecutive meetings are not quorate 8. Proposals open for more than 12 months must be taken to vote -9. Final decisions recorded in [Architectural Decision Records](https://adr.github.io/) +9. Final decisions recorded in [Architectural Decision Records](adr/) 10. CPO may reopen issues or review decisions in consultation with PSC, TASC, SLC, or GA4GH Executive Committee ### **Step 4: Dissemination** diff --git a/adr/template.md b/adr/template.md new file mode 100644 index 0000000..adc064d --- /dev/null +++ b/adr/template.md @@ -0,0 +1,74 @@ +# TASC-ADR-XXX: [Short Decision Title] + +**Date:** YYYY-MM-DD | **Status:** [Proposed | Accepted | Deprecated | Superseded] + +**Deciders:** [List key decision-makers] +**Keywords:** [e.g., maturity-model, versioning, governance, api, security] +**Work Streams Impacted:** [e.g., Cloud, Discovery, DaMaSC] +**Products Affected:** [e.g., Beacon v2, TRS, Refget] + +--- + +## Context + +[Describe the problem and why a decision is needed. What forces are at play?] + +**The Problem:** +- [Describe the issue requiring a decision] + +**Alternatives Considered:** + +1. **[Alternative 1]** + - ✅ [Pro] + - ❌ [Con] + +2. **[Alternative 2]** + - ✅ [Pro] + - ❌ [Con] + +3. **[Chosen Alternative]** + - ✅ [Pro] + - ✅ [Another pro] + +--- + +## Decision + +[State the decision clearly. Use active voice and full sentences.] + +**Key Points:** +- [Point 1] +- [Point 2] + +--- + +## Consequences + +### Positive + +✅ [Benefit 1] +✅ [Benefit 2] + +### Negative + +❌ [Cost/drawback 1] +❌ [Cost/drawback 2] + +### Risks & Mitigations + +**Risk:** [Description] +- **Mitigation:** [How to address] + +--- + +## References + +- **Full Policy:** [Link to detailed documentation] +- **Related ADRs:** [Links to related decisions] +- **Examples:** [Links to implementations] + +--- + +## Notes + +[Any additional context, rationale, or future considerations] diff --git a/governance/TASC_Document_Standards.md b/governance/TASC_Document_Standards.md new file mode 100644 index 0000000..5b307b0 --- /dev/null +++ b/governance/TASC_Document_Standards.md @@ -0,0 +1,222 @@ +# TASC Document Standards + +**Document Type**: Governance Charter +**Document ID**: TASC-GOV-02 +**Source**: TASC +**Title**: TASC Document Standards: Format, Placement, and Communication Standards for TASC Outputs +**Related GitHub issues**: [#76](https://github.com/ga4gh/TASC/pull/76) +**Raised by**: Mamana Mbiyavanga (TASC) +**Authors**: Mamana Mbiyavanga, Andy Yates, Sasha Siegel, Jinny Park +**Date**: 2026-02-01 +**Status**: Draft +**Keywords**: governance, document-standards, templates, numbering, placement, communication +**Work Streams Impacted**: All work streams + +## Abstract + +This document formalizes the format, placement, and communication standards for all TASC outputs. It establishes two document types (living policies and immutable decision records), standard metadata templates, a numbering convention, the GitHub repository as the canonical source of truth, and a communication workflow to ensure approved outputs reach the GA4GH community. + +## Table of contents + +- [Background](#background) +- [Document Types](#document-types) +- [Metadata Templates](#metadata-templates) +- [Numbering Convention](#numbering-convention) +- [Repository Structure](#repository-structure) +- [Document Lifecycle](#document-lifecycle) +- [Communication Workflow](#communication-workflow) +- [Website and Airtable Integration](#website-and-airtable-integration) +- [References](#references) +- [Contributors](#contributors) + +## Background + +TASC has produced a growing number of approved outputs -- recommendations, governance charters, and architectural decision records. Without standardized format and placement, outputs were scattered across Google Docs, GitHub, emails, and meeting notes. Stakeholders could not easily find or reference decisions, leading to duplication, version confusion, and inconsistent formats across authors and time periods. New members had no efficient way to catch up on past decisions. + +This document formalizes the structure that has been adopted in the `ga4gh/TASC` GitHub repository, drawing on practices from other standards organizations (HL7 FHIR, W3C, IETF) that separate living specifications from decision rationale. + +## Document Types + +TASC outputs fall into two categories with distinct characteristics: + +### Recommendations and Governance Documents (Living) + +| Attribute | Description | +|-----------|-------------| +| **Purpose** | Ongoing rules, procedures, standards, and guidance | +| **Nature** | Living, prescriptive, comprehensive | +| **Length** | Typically 10-50+ pages | +| **Scope** | Broad domain or process | +| **Key Question** | "What are the rules?" | + +These documents define processes and standards that apply broadly and MAY evolve over time through a formal revision process. Examples include the API Pagination Guide (GA4GH-REC-01) and the TASC Governance Charter (TASC-GOV-01). + +### Architectural Decision Records (Immutable) + +| Attribute | Description | +|-----------|-------------| +| **Purpose** | Single decision + rationale | +| **Nature** | Immutable, explanatory, focused | +| **Length** | 1-2 pages | +| **Scope** | Single architectural choice | +| **Key Question** | "Why did we decide this way?" | + +ADRs follow the Michael Nygard template and capture the context, alternatives considered, decision made, and consequences. Once accepted, ADRs MUST NOT be edited. If a decision changes, a new ADR supersedes the original. + +### When to Use Which + +- **Write a Recommendation/Governance document when**: Defining ongoing rules, procedures, or standards that apply broadly and may evolve. +- **Write an ADR when**: The decision could have gone another way and the rationale matters for future reference. + +The policy tells the process. The ADR records when that process was used to make a specific choice. + +## Metadata Templates + +### Recommendation / Governance Metadata Header + +All recommendations and governance documents MUST include the following metadata header: + +```markdown +**Source**: TASC +**Recommendation**: GA4GH-REC-XX (or **Document ID**: TASC-GOV-XX) +**Title**: [Full descriptive title] +**Related GitHub issues**: [#N](https://github.com/ga4gh/TASC/issues/N) +**Raised by**: [Name (Work Stream/Role)] +**Authors**: [Author names, comma-separated] +**Date:** YYYY-MM-DD +**Status:** [Draft | Approved | Superseded] +**Keywords**: [comma-separated keywords] +**Work Streams Impacted**: [affected work streams] +**Products Affected**: [affected GA4GH products] +``` + +### ADR Metadata Header + +All ADRs MUST include the following metadata header and follow the Nygard template structure: + +```markdown +# TASC-ADR-XXX: [Short Decision Title] + +**Date:** YYYY-MM-DD | **Status:** [Proposed | Accepted | Deprecated | Superseded] +**Deciders:** [List key decision-makers] +**Keywords:** [comma-separated keywords] +**Work Streams Impacted:** [affected work streams] +**Products Affected:** [affected GA4GH products] + +## Context +## Decision +## Consequences +``` + +### Metadata Field Definitions + +| Field | Description | +|-------|-------------| +| **Source** | Always "TASC" | +| **Raised by** | The person and work stream/role that originally raised the issue (recommendations only) | +| **Authors** | Those who wrote the document | +| **Deciders** | Those who made the decision (ADRs only) | +| **Keywords** | Searchable tags for discoverability (e.g., api, pagination, governance, maturity-model) | +| **Work Streams Impacted** | GA4GH work streams affected by the document | +| **Products Affected** | Specific GA4GH products impacted | +| **Status** | Current state: Draft, Approved/Accepted, Superseded/Deprecated | + +Each metadata line MUST end with two trailing spaces for proper Markdown line breaks. + +## Numbering Convention + +| Document Type | ID Format | Example | +|---------------|-----------|---------| +| Recommendations | `GA4GH-REC-XX` | GA4GH-REC-01 | +| Governance documents | `TASC-GOV-XX` | TASC-GOV-01 | +| Architectural Decision Records | `TASC-ADR-XXX` | TASC-ADR-001 | + +- Recommendations use the `GA4GH-` prefix as they represent organization-wide guidance. +- Governance documents and ADRs use the `TASC-` prefix as they are scoped to the sub-committee. +- Numbers are zero-padded and assigned sequentially. + +## Repository Structure + +The `ga4gh/TASC` GitHub repository is the canonical source of truth for all TASC outputs: + +```text +ga4gh/TASC/ +├── README.md +├── adr/ +│ ├── README.md (index table) +│ ├── template.md +│ └── NNN-short-title.md (accepted ADRs) +├── governance/ +│ └── TASC-GOV-XX_Title.md +├── recommendations/ +│ ├── template.md +│ └── [Recommendation Title].md +├── drafts/ (work in progress) +├── service-info/ +│ └── ga4gh-service-info.json +└── data/ (supporting materials, not committed) +``` + +- **`adr/`** -- Immutable decision records. ADR files use the naming convention `NNN-short-kebab-title.md` with zero-padded 3-digit numbers. +- **`governance/`** -- TASC operating charters and leadership rules. +- **`recommendations/`** -- Approved policy documents and guidance. +- **`drafts/`** -- Work-in-progress documents that graduate to their respective directories upon approval. +- **`service-info/`** -- The GA4GH Service Info Type Registry. + +## Document Lifecycle + +Documents follow the TASC charter's four-step decision-making process: + +1. **GitHub Issue** -- Anyone in the GA4GH community can raise a technical alignment concern. +2. **Discussion** -- TASC meetings and asynchronous discussion on GitHub/Slack. +3. **Draft** -- Work in progress, placed in `drafts/`. +4. **Review / Vote** -- 70% approval threshold per the TASC Governance Charter (TASC-GOV-01). +5. **Approved** -- Placed in the appropriate directory: + - Recommendations go to `recommendations/` + - ADRs go to `adr/` + - Governance documents go to `governance/` + +## Communication Workflow + +When a new output is approved, the following communication steps SHOULD be followed: + +| Step | Action | Channel | +|------|--------|---------| +| 1 | Merge PR | GitHub repository (source of truth) | +| 2 | Update Airtable | Metadata registry | +| 3 | Post announcement | TASC mailing list + Slack #tasc | +| 4 | Update website | GA4GH website cards | +| 5 | Present at GA4GH Connect | If the output is significant | + +This ensures no approved output "goes dark" -- every output reaches the people who need to know. + +## Website and Airtable Integration + +All approved outputs SHOULD appear as cards on the GA4GH website, linking back to the GitHub source document. The Airtable metadata registry SHOULD contain the following fields for each output: + +| Field | Description | +|-------|-------------| +| Title | Document name | +| Description | Brief summary | +| Alignment Scope | GA4GH alignment area | +| Work Streams | Affected work streams | +| Products | Related GA4GH products | +| Tags | Searchable keywords | +| Directive (Y/N) | Mandatory vs. guidance | +| Link | URL to GitHub source | + +## References + +- [TASC Governance and Leadership Charter (TASC-GOV-01)](TASC_Governance_and_Leadership_Approved_240825.md) +- [Michael Nygard's ADR template](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions) +- [TASC Placement Proposal Presentation](https://docs.google.com/presentation/d/1zc4VeoZVaU2fIr6dP9VSFhHxIceWzglt/edit) (February 2026) +- [GA4GH TASC GitHub Project Board](https://github.com/orgs/ga4gh/projects/9/views/1) + +## Contributors + +| Name | Organisation | +|------|-------------| +| Mamana Mbiyavanga | University of Cape Town / TASC Co-lead | +| Andy Yates | EMBL-EBI / TASC Co-lead | +| Sasha Siegel | GA4GH / CPO | +| Jinny Park | GA4GH / Technical Team | diff --git a/governance/TASC_Governance_and_Leadership_Approved_240825.md b/governance/TASC_Governance_and_Leadership_Approved_240825.md new file mode 100644 index 0000000..2861b8c --- /dev/null +++ b/governance/TASC_Governance_and_Leadership_Approved_240825.md @@ -0,0 +1,141 @@ +# TASC Governance and Leadership Charter + +**Document Type**: Governance Charter +**Document ID**: TASC-GOV-01 +**Source**: TASC +**Title**: TASC Governance and Leadership Charter +**Related GitHub issues**: [#65](https://github.com/ga4gh/TASC/issues/65) +**Authors**: TASC Leadership +**Date**: 2024-08-25 +**Status**: Approved +**Keywords**: governance, charter, leadership, decision-making +**Work Streams Impacted**: All work streams + +## Table of contents + +- [Abstract](#abstract) +- [Mission Statement and Scope](#mission-statement-and-scope) +- [TASC Structure](#tasc-structure) + - [Membership](#membership) +- [Group Meeting Format](#group-meeting-format) +- [Decision-Making Process](#decision-making-process) + - [Step 1: Collecting and Evaluating Issues](#step-1-collecting-and-evaluating-issues) + - [Step 2: Working on a Solution](#step-2-working-on-a-solution) + - [Step 3: Making Decisions](#step-3-making-decisions) + - [Step 4: Dissemination](#step-4-dissemination) +- [Additional points](#additional-points) +- [Contributors](#contributors) + +## Abstract + +This charter defines the governance structure, membership, decision-making processes, and operational procedures of the GA4GH Technical Alignment Sub-Committee (TASC). It establishes TASC's mission to harmonize GA4GH technical products, outlines voting and non-voting membership roles, defines meeting formats (open/closed sessions), and specifies the four-step decision-making process from issue collection through dissemination. The charter includes membership term limits, voting thresholds (70% approval), and provisions for reopening decisions through CPO consultation with stakeholder committees. + +## Mission Statement and Scope + +The mission of the Technical Alignment Sub-Committee (TASC) of the GA4GH Product Steering Committee (PSC) is to aid the harmonisation of GA4GH's technical products to ensure they can be used together effectively. TASC provides outputs and decisions to create internal consistency and technical alignment across GA4GH Work Streams and deliverables. + +While TASC functions as a central decision-making body, all of its decisions are informed by relevant stakeholders. The TASC outputs, including all decisions, are regularly and openly communicated back to the GA4GH community. Where appropriate, TASC will action product development related to technical alignment within its structure when a need is identified and would benefit the GA4GH community, the nature of the work does not naturally fit within one of the workstreams, the work requires significant cross-workstream development, and where sufficient interest and resources exist to develop the solution. + +## TASC Structure + +TASC is responsible for the development and oversight of solutions (including directives and products) to technical alignment issues across GA4GH workstreams. The TASC member network consists of participants to relay technical alignment issues and insights based on product knowledge and expertise, as well as the following voting members, who will make and uphold decisions regarding solutions in the best interest of GA4GH products collectively. This structure can be evaluated and updated on an as-needed basis and can be triggered by the CPO when the need arises. + +### Membership + +The voting members of TASC are: + +* **The TASC Leadership** is composed of two volunteer co-leads who are responsible for convening the group, setting its agenda, facilitating decisions, and serving as the central point of communication for stakeholders. Agenda-setting includes identifying and prioritizing the issues to be addressed by the group. The CPO decides who holds the co-lead positions. +* **The Chief Product Officer (CPO)** ensures that the team creates GA4GH-wide technical alignment and that the needs of GA4GH are appropriately considered in any TASC outputs +* **Work Stream Representatives.** Two volunteer representatives from each technical Work Stream and Data Security–preferably the Work Stream Co-leads–will provide knowledge and insight for their respective GA4GH products and communicate as liaisons. If they are not the representatives, the Work Stream Co-leads will select these representatives. + +The non-voting members (advisory/contributors) are: + +* Invited representative(s) from the **Strategic Leadership Committee (SLC)** to provide a strategic lens to TASC and represent TASC initiatives at SLC meetings +* Invited experts from the GA4GH community +* Other appropriate members of the **GA4GH staff, including the GA4GH Tech Team.** + +TASC co-leads and representatives will be expected to serve renewable two-year terms. + +## Group Meeting Format + +1. The Technical Alignment Sub-Committee (TASC) meets monthly, in either *open* or *closed* sessions, to: + 1. Review unresolved issues in the TASC GitHub repository (*open*) + 2. Discuss potential unmet needs that could result in the generation of a new issue being added to the TASC GitHub repository (*open*) + 3. Discuss and review any work being conducted within TASC’s remit, for example, regular updates from TASC working groups (*open*) + 4. Review all GA4GH technical standards that have been submitted to the PSC for approval (*closed*) +2. Any topics requiring sensitive discussion or voting will be in closed session. +3. For asynchronous discussion of issues, members can use + 1. Mailing List: Online and in-person meetings will be announced via email using this address: [tasc@ga4gh.org](mailto:tasc@ga4gh.org) + 2. GitHub: Communication regarding technical contributions and questions is expected in the GitHub repository, found at [https://github.com/ga4gh/TASC](https://github.com/ga4gh/TASC) + 3. Slack: Additional relevant discussions will happen on the GA4GH Slack Workspace. Use the \#tasc channel. + +## Decision-Making Process + +### **Step 1: Collecting and Evaluating Issues** + +1. Any member of the GA4GH community may raise technical alignment concerns and propose technical alignment solutions as [new Issues in the TASC GitHub repository](https://github.com/ga4gh/TASC/issues). On an operational note, pushing to the [ga4gh/TASC](https://github.com/ga4gh/TASC) repo will be restricted to the TASC co-leads; everyone else will use a fork and pull request approach. +2. The TASC co-leads and CPO will triage issues. New issues will be presented to the wider TASC group at the next available meeting. The wider TASC group will decide whether to accept IN SCOPE issues or close OUT OF SCOPE issues with a comment on the proposed GitHub issue based on the following criteria: + 1. Issues that are IN SCOPE should have multiple invested adopters and MUST do one or more of the following: + 1. Aid the harmonisation of aspects of GA4GH's various products, including the development, documentation, and deployment of said products, e.g., supporting schema alignment + 2. Lower the barriers to organisation-wide harmonisation between GA4GH products. + 3. Produce directives, policies, and other documents that may enable greater harmonisation, interoperability, or use of specific GA4GH standards. + 2. Issues may be closed with a comment if they: + 1. are not feasible, e.g., too onerous to manage + 2. require significant development, and there is a lack of sufficient resources/interest to develop within TASC + 3. impose significant operational requirements on existing teams +3. IN SCOPE issues that would generate outputs that require sufficient levels of cross-work stream development, that can’t realistically be developed in the conventional work streams, may be developed at the discretion of TASC and/or PSC + 1. Outputs developed and implemented for internal use, including enterprise tools and infrastructure, WILL NOT be subject to the product development and approval process + 2. Directives and policy statements issued by TASC WILL NOT be subject to the product development and approval process and MUST be reviewed and approved by PSC + 3. External-facing products (e.g., a standard specification) WILL be subject to the product development and approval process + 1. Other outputs will be reviewed and governed as per the discretion of TASC + +### **Step 2: Working on a Solution** + +1. Once an issue has been brought to TASC, it will be added to the [TASC project board](https://github.com/orgs/ga4gh/projects/9/views/1?filterQuery=-status%3ADone) and scoped accordingly, and any proposals that are deemed to be out of scope will be documented with the reason to maintain a decision log and retired. Any issue can be reactivated should interest and resources become available in the future. +2. For an issue to be in-scope and prioritised, two TASC members (product owners) must be assigned to the issue. In the event that the most appropriate team to work on a solution is a subgroup, the issue will be actioned there. We will also seek adopters to help contribute towards solutions +3. Owners research the issue, talking to relevant stakeholders and summarising their findings in a brief memo (as a Google document or GitHub repository). +4. GA4GH staff makes the memo available for open comment on the GA4GH website. +5. Any outputs that are proposed for development following the stakeholder consultation are similarly brought to TASC for discussion. Any output approved for development may need larger teams to work on the issue, and the product owner is accountable for creating the development team needed for the work, including the formation of small TASC forces or larger subgroups. +6. All GA4GH active contributors, including TASC members, have a chance to weigh in on the memo +7. ALL TASC members are expected to have read the memo prior to the scheduled vote + +### **Step 3: Making Decisions** + +1. To reach a final decision, the two issue owners (see 2.2) must seek a vote from TASC members. The vote will be conducted asynchronously with materials presented at a TASC meeting and seeking to close voting by the next scheduled meeting, i.e., one calendar month. +2. Notes on Voting + 1. Members who vote will be given three options: 1\) approve, 2\) reject, 3\) abstain. + 2. The approval threshold is 70% of votes of the total eligible members with a “yes” vote. + 3. The decision-making and voting process aims to + 1. Ascertain a high level of consensus rather than unanimity + 2. Ensure those who have opinions, raised objections, or contributed to development have an opportunity to share their thoughts + 3. Ensure relevant GA4GH processes have been followed correctly + 4. Members must ensure that products/outcomes (e.g., data models and specs): + 1. do what they're designed to do without unduly constraining future development + 2. are intentionally aligned with other specs at key touchpoints + 3. do not create unnecessary barriers to interoperability +4. Where approval is not reached through multiple rounds of votes, TASC Leadership, including the CPO, will make a final decision. +5. Leadership can take decisions if two consecutive TASC meetings are not quorate — assuming efforts have been made between meetings to collect votes asynchronously +6. Votes can be requested for approval over email when an issue is simple to close or when a decision must be taken between meetings. +7. Proposals that have been open for more than 12 months must be taken to vote; if the proposal is rejected, it may be revised and resubmitted as a new issue. +8. While TASC discussions will occur within GitHub issues, final decisions will be recorded in an [Architectural Decision Record](https://adr.github.io/) (ADR) which will be made publicly available at the TASC GitHub repository. +9. While TASC decisions are intended to be conclusive, the CPO, in consultation with the PSC, TASC, SLC, or GA4GH Executive Committee, may determine that the issue should be reopened or the decision should be reviewed. The CPO will regularly update the relevant GA4GH committees on progress within any TASC developments to retain alignment with the GA4GH community and its objectives/needs. + +### **Step 4: Dissemination** + +1. Decisions will be included in the GA4GH communications, reported at the PSC meetings, and published to the GA4GH website. +2. TASC members and Technical Programme Managers are responsible for maintaining awareness of these decisions within their work streams. +3. Content-specific indices, repositories, etc., will be created as needed to facilitate the organisation and dissemination of decisions and common components. + +## Additional points + +* TASC members should help socialise issues by discussing those relevant to their work streams. +* The proposal here may require an adjustment to the Product Development and Approval Process. Concerning the creation of products from TASC and the use of TASC to review technical products going through approval +* TASC outputs should be positioned as GA4GH best practice, where we expect conformance on new GA4GH products, but do not expect backwards compatibility with historical outputs +* Any centralised registries or similar structures, if needed, will be managed by GA4GH staff +* Community will be empowered to use TASC and encourage submission of TASC issues, but we will need a mechanism to achieve this +* TASC will use the Open Calls mechanism from [ga4gh.org](http://ga4gh.org) to raise awareness of solutions requiring input +## Contributors + +- TASC Leadership +- GA4GH Product Steering Committee +- GA4GH Work Stream Representatives diff --git a/recommendations/API pagination guide.md b/recommendations/API pagination guide.md index 1eb703f..27a3690 100644 --- a/recommendations/API pagination guide.md +++ b/recommendations/API pagination guide.md @@ -1,12 +1,16 @@ # API pagination guide recommendation **Source**: TASC -**Recommendation**: GA4GH-01 +**Recommendation**: GA4GH-REC-01 **Title**: API pagination guide -**Related GitHub issues**: [TASC\#29](https://github.com/ga4gh/TASC/issues/29) -**Author**: Mamana Mbiyavanga, Andy Yates +**Related GitHub issues**: [#29](https://github.com/ga4gh/TASC/issues/29) +**Raised by**: Jeremy Adams (Cloud) +**Authors**: Mamana Mbiyavanga, Andy Yates **Date:** 2024-11-25 -**Status:** Approved +**Status:** Approved +**Keywords**: api, pagination, interoperability, cursor, offset +**Work Streams Impacted**: Discovery, Cloud, LSG +**Products Affected**: TRS, Beacon v2, Data Connect ## Abstract @@ -198,19 +202,19 @@ Because pagination forces data streaming over multiple requests, there is a poss ### Use case random seeking within a result set -**Use page-based pagination** +**Use page-based pagination** Random-seeking is a functionality normally requested in human interaction interfaces or where more flexibility needs to be afforded to a client. In the first example, consider an interface which displays the results of a search. Using random-seeking allows a client to skip an arbitrary number of pages ahead in the results or configure the number of records displayed per page. By using random-seeking a product can afford future clients the ability to configure their paging requirements without enforcing a predetermined policy on all clients. It also allows a client to walk back and forth along a result set. Many underlying data querying technologies implement random seeking (e.g. Solr, ElasticSearch, MongoDB) and as such adoption is easy. ### Use case parallel block retrieval -**Use page-based pagination** +**Use page-based pagination** To speed up results retrieval, a client may wish to split results retrieval into multiple parallel requests. A client can construct URLs representing a block of data to retrieve based on a specified size and page into the results set. Parameters must be predictable and allow random seeking within a response (see above use-case). ### Use case streaming large result sets -**Use page-based pagination** +**Use page-based pagination** When encountering the “Deep Paging” issue i.e. where the number of results is a large number, many data querying technologies switch to a token-based system where the last result returned is remembered to ensure pagination consistency and predictability. Results processing is normally a one-way process i.e. you cannot request a prior page. diff --git a/recommendations/GA4GH product owner description.md b/recommendations/GA4GH product owner description.md new file mode 100644 index 0000000..18e1241 --- /dev/null +++ b/recommendations/GA4GH product owner description.md @@ -0,0 +1,57 @@ +# GA4GH Product Owners: Integrating Software Product Management Principles + +**Source**: TASC +**Recommendation**: GA4GH-REC-03 +**Title**: GA4GH Product Owners: Integrating Software Product Management Principles into GA4GH Standards & Policies Development +**Related GitHub issues**: [#49](https://github.com/ga4gh/TASC/issues/49), [#64](https://github.com/ga4gh/TASC/issues/64), [#74](https://github.com/ga4gh/TASC/issues/74) +**Raised by**: Jonathan Lawson (DURI) +**Authors**: Jonathan Lawson, Andy Yates +**Date:** 2025-07-18 +**Status:** Approved +**Keywords**: governance, product-management, work-streams +**Work Streams Impacted**: All work streams +**Products Affected**: None directly + +## Table of contents + +- [Abstract](#abstract) +- [Background: Fundamentals of Software Product Management](#background-fundamentals-of-software-product-management) + - [Applying Product Management Principles to GA4GH Product Development](#applying-product-management-principles-to-ga4gh-product-development) + - [Conclusion](#conclusion) +- [Contributors](#contributors) + +## Abstract + +This recommendation establishes the role and principles of Product Owners for GA4GH standards and policy development. It applies software product management discipline to ensure standards are developed strategically, aligned with user needs, and avoid scope creep. The document defines Product Owner responsibilities, reporting structures, term limits, and oversight mechanisms to ensure each GA4GH standard has dedicated leadership guiding its development from conception to long-term maintenance. + +## **Background: Fundamentals of Software Product Management** + +Software Product Management is a critical discipline that ensures software products are developed and evolve in alignment with strategic goals and user needs. It serves as the hub that connects the product’s go-to-market strategy, user needs, and technology development. It also ensures that an organization builds the **right product** for the **right people** at the **right time**. The following are product management elements and principles to be implemented for standards and policies development by the Global Alliance for Genomics and Health (GA4GH). + +**Key Elements of Effective Software Product Management:** + +1. **The GA4GH Product Owner (PO)** plays a key role in this process, acting as the central point of contact for a product. The core function of the PO is to own the product's success from conception to launch and beyond. The PO is responsible for setting a clear product vision and roadmap, gathering and prioritizing feature requests, and guiding the product through iterative development cycles with a technically diverse Scrum team. +2. **Product Vision and Roadmap:** The PO defines and communicates a clear vision for the product's future, ensuring alignment with strategic objectives. This vision is translated into a roadmap that outlines the product's development trajectory over time. +3. **Feature Prioritization and Iterative Development:** Effective product management involves taking in feature requests from various stakeholders but making strategic decisions about what gets built based on value, feasibility, and impact. Iterative development allows for continuous improvement and adaptability to changing needs and environments. +4. **Dedicated Leadership and Focus:** A Product Owner ensures the product does not become an "albatross"—a cumbersome, unfocused tool resulting from fragmented decision-making or saying "yes" to every request. By having a thoughtful point of contact, the product remains strategically aligned and avoids the pitfalls of haphazard multiparty design or overly flat authority structures. +5. **Cross-Functional Collaboration:** The PO works closely with a diverse Scrum team, which includes developers, designers, testers, and other stakeholders, to ensure the product is developed efficiently and meets user needs. The PO's role as a liaison helps to align technical, business, and user perspectives. + +### **Applying Product Management Principles to GA4GH Product Development** + +Standards organizations that develop and implement new policies and standards for science can benefit significantly from the principles of software product management. Similar to software products, each standard or policy is a "product" that requires thoughtful development, iteration, and strategic direction. + +**Assigning Product Owners for GA4GH Products on Standards & Policies** + +1. **Product Owners for GA4GH Products:** Each standard or policy should have at least one dedicated Product Owner responsible for guiding its development, ensuring strategic alignment, and preventing scope creep. The Product Owner will work closely with Workstream Leads to set the roadmap and direction for both the individual standards and the broader workstream. At times, Workstream Leads may serve as Product Owners. The product owners(s) should be set upon product approval. +2. **Reporting and Collaboration Structure:** Product Owners will report to the Workstream Leads and collaborate with them to prioritize and sequence product development and revisions. This will provide a structured approach to developing standards and ensure that policies evolve in a way that is consistent with the organization's mission and goals. The Workstream Leads will remain accountable to GA4GH. +3. **Term-Limited Roles:** To ensure fresh perspectives and prevent stagnation, Product Owner roles should be term-limited (e.g., 24 months), with an option to renew. Limited terms guarantee that the standards development process remains dynamic and responsive to changing scientific landscapes and community needs while giving Product Owners ample time to deliver value in their role. Whether there is a single product owner or two owners per product is up to the discretion and needs of individual work streams, and ultimately decided by work stream co-leads. +4. **Oversight and Accountability:** These roles will be served at the discretion of the work stream leads, ensuring accountability and alignment with the organization's broader strategic objectives. Should there be challenges that require organisational leadership, the CPO can intervene to support needs as they arise. + +### **Conclusion** + +By integrating the fundamentals of software product management into the development of scientific standards and policies, the organization can ensure that each standard is developed thoughtfully, strategically, and in alignment with the needs of the scientific community. This structured approach will help prevent the proliferation of unfocused or conflicting standards, ultimately leading to more coherent and impactful policy development. + +## Contributors + +- TASC Leadership +- GA4GH Product Steering Committee diff --git a/recommendations/Service Info Type Registry Guide.md b/recommendations/Service Info Type Registry Guide.md new file mode 100644 index 0000000..c8e82c9 --- /dev/null +++ b/recommendations/Service Info Type Registry Guide.md @@ -0,0 +1,284 @@ +# Service Info Type Registry Guide recommendation + +**Source**: TASC +**Recommendation**: GA4GH-REC-02 +**Title**: Service Info Type Registry Guide +**Related GitHub issues**: [#1](https://github.com/ga4gh/TASC/issues/1), [#16](https://github.com/ga4gh/TASC/issues/16), [#67](https://github.com/ga4gh/TASC/issues/67) +**Raised by**: Melissa Konopko (Technical Team) +**Authors**: Jeremy Adams, John Marshall, Mamana Mbiyavanga +**Date:** 2025-01-28 +**Status:** Approved +**Keywords**: service-info, registry, discovery, naming +**Work Streams Impacted**: All work streams +**Products Affected**: All GA4GH API products + +## Abstract + +This recommendation defines the governance, structure, and lifecycle management of the GA4GH Service Info Type Registry, which maintains a centralized registry of standardized service type identifiers for GA4GH API servers. The registry enables service discovery within GA4GH networks by providing a consistent naming system using a two-part identifier structure (group and artifact). This document describes the process for registering new service types, the required naming conventions, lifecycle management of registry entries, and guidance for implementers. The recommendation establishes TASC as the governing body for registry additions and defines processes that ensure backward compatibility and prevent identifier collisions. + +## Table of contents + +- [Recommendation](#recommendation) +- [Purpose of the Service Info Type Registry](#purpose-of-the-service-info-type-registry) +- [Service type structure](#service-type-structure) +- [Registration process](#registration-process) +- [Registry entry format](#registry-entry-format) +- [Lifecycle management](#lifecycle-management) +- [Implementation guidance](#implementation-guidance) +- [Security considerations](#security-considerations) +- [Use cases](#use-cases) +- [Considerations](#considerations) +- [References](#references) +- [Contributors](#contributors) + +## Recommendation + +GA4GH services MUST register their service type identifiers in the centralized Service Info Type Registry maintained by TASC. Service types MUST use the standardized two-part naming structure consisting of a group identifier (typically "org.ga4gh") and an artifact identifier in kebab-case format. New service type registrations MUST follow the submission and review process defined in this document. Registry entries MUST NOT be deleted; deprecated entries SHOULD be marked with appropriate status annotations. Implementations MUST expose their service type information through the Service Info endpoint as defined in the GA4GH Service Info specification. This recommendation applies to new GA4GH products and existing products MAY adopt it through coordination with TASC. + +## Purpose of the Service Info Type Registry + +The Service Info Type Registry provides a centralized, authoritative source of standardized identifiers that enable GA4GH API servers to declare their capabilities in a machine-readable format. This allows clients, network operators, and monitoring tools to discover available services without manual configuration or documentation lookups. By maintaining a single registry, GA4GH ensures consistency across implementations, prevents identifier collisions, and provides a clear governance process for adding new service types. The registry supports network-level orchestration, automated service discovery, capability negotiation, and monitoring of GA4GH service ecosystems. It serves as the foundation for building federated networks of GA4GH services that can interoperate seamlessly. + +## Service type structure + +### Group identifier + +The group identifier represents the organization or namespace that defines and maintains the service type. For GA4GH standardized services, the group identifier MUST be "org.ga4gh". Organizations implementing custom extensions or non-standard services MAY use their own group identifier following reverse domain notation (e.g., "org.example" for example.org). + +### Artifact identifier + +The artifact identifier specifies the particular API or service type within the group namespace. Artifact identifiers MUST use kebab-case formatting: lowercase letters and numbers separated by hyphens. The artifact name SHOULD be concise, descriptive, and reflect the service's primary function. For versioned APIs, the version MAY be included in the artifact identifier (e.g., "beacon-v2") or handled through separate versioning fields. + +### Naming constraints + +Artifact identifiers MUST conform to the following regular expression pattern: + +``` +^([a-z][a-z0-9]*)(-[a-z0-9]+)*$ +``` + +This pattern enforces: +- Must start with a lowercase letter +- May contain lowercase letters and digits +- May include hyphens as separators (but not at the start or end) +- No uppercase letters, underscores, or special characters + +Valid examples: `tool-registry-service`, `beacon-v2`, `drs` +Invalid examples: `ToolRegistry`, `beacon_v2`, `DRS`, `-service` + +## Registration process + +### Submitting new service types + +Product developers MUST submit new service type registrations through GitHub pull requests to the TASC repository. The submission process is: + +1. Fork the GA4GH TASC repository +2. Modify the `service-info/ga4gh-service-info.json` file to add the new entry +3. Maintain alphabetical ordering of entries by artifact identifier +4. Submit a pull request with a clear description of the service type +5. Notify the GA4GH Secretariat or TASC Force via appropriate communication channels + +Submitters SHOULD provide context about the service's purpose, its relationship to existing GA4GH standards, and expected adoption timeline. + +### Review and approval workflow + +TASC reviews service type registration requests during regular meetings or asynchronously through GitHub. The review process evaluates: + +- Naming compliance with registry conventions +- Uniqueness and lack of collision with existing identifiers +- Alignment with GA4GH principles and standards +- Completeness and clarity of submission + +TASC MAY request revisions, suggest alternative naming, or require additional documentation. Contributors MUST respond to review feedback to advance the registration. Approvals are documented through pull request merge and recorded in TASC meeting minutes. + +### Coordination with product approval + +Service type registration SHOULD be coordinated with the GA4GH product approval process. Products undergoing standards approval SHOULD submit service type registrations early in the approval cycle to ensure consistency between the service type identifier and the final product specification. For products already approved, service type registration MAY occur retrospectively but SHOULD reference the approved product specification. + +## Registry entry format + +### Required fields + +Each registry entry MUST include: + +- **type**: An object containing: + - **group**: The group identifier (typically "org.ga4gh") + - **artifact**: The artifact identifier in kebab-case + +```json +{ + "type": { + "group": "org.ga4gh", + "artifact": "service-name" + } +} +``` + +### Optional fields + +Entries MAY include additional metadata: + +- **version**: The API version string if not encoded in the artifact identifier +- **status**: Lifecycle status ("active", "superseded", "withdrawn") +- **note**: Human-readable description or deprecation notice +- **superseded_by**: Reference to the replacement service type (for deprecated entries) +- **documentation_url**: Link to official service documentation + +```json +{ + "type": { + "group": "org.ga4gh", + "artifact": "old-service" + }, + "status": "superseded", + "note": "Replaced by new-service v2.0", + "superseded_by": { + "group": "org.ga4gh", + "artifact": "new-service" + } +} +``` + +### Example entries + +Active service type: +```json +{ + "type": { + "group": "org.ga4gh", + "artifact": "tool-registry-service" + } +} +``` + +Deprecated service type: +```json +{ + "type": { + "group": "org.ga4gh", + "artifact": "legacy-api" + }, + "status": "withdrawn", + "note": "Service discontinued as of 2024-06-01" +} +``` + +## Lifecycle management + +### Active entries + +Active service types represent currently supported GA4GH APIs. These entries remain in the registry indefinitely and serve as the authoritative reference for implementers. Updates to active entries SHOULD be minimal and limited to adding optional metadata fields. Changes to the core type identifiers (group and artifact) MUST NOT occur once an entry is registered. + +### Deprecated entries + +When a service type is superseded or withdrawn, the entry MUST remain in the registry with appropriate status annotations. This preserves the historical record and prevents identifier reuse that could cause confusion. Deprecated entries MUST include: + +- A **status** field set to "superseded" or "withdrawn" +- A **note** field explaining the deprecation reason +- For superseded services, a **superseded_by** reference to the replacement + +Servers implementing deprecated service types SHOULD continue to report them in their Service Info responses to maintain backward compatibility with older clients. + +### Versioning considerations + +Service type identifiers MAY encode version information in the artifact name (e.g., "beacon-v2") or through separate version fields. When major API revisions occur that break backward compatibility, implementers SHOULD register a new service type entry rather than modifying the existing one. This allows servers to explicitly signal which API versions they support and enables clients to make informed compatibility decisions. + +## Implementation guidance + +### Server implementation + +Servers MUST implement the GA4GH Service Info endpoint as defined in the Service Info specification. The endpoint MUST return a JSON response containing the service type information from the registry. Minimal implementation: + +```json +{ + "id": "org.example.server-001", + "name": "Example GA4GH Server", + "type": { + "group": "org.ga4gh", + "artifact": "tool-registry-service" + }, + "organization": { + "name": "Example Organization" + }, + "version": "1.0.0" +} +``` + +Servers implementing multiple GA4GH APIs MAY report multiple service types through additional fields or repeated type entries, depending on the Service Info specification version. + +### Client discovery + +Clients SHOULD query the Service Info endpoint to determine server capabilities before making API requests. This enables: + +- Automatic detection of supported API versions +- Graceful degradation when servers don't support expected features +- Dynamic routing in federated networks +- Capability-based service selection + +Clients SHOULD NOT hardcode assumptions about server capabilities and SHOULD handle unknown service types gracefully. + +### Multi-service servers + +Servers hosting multiple GA4GH APIs SHOULD expose multiple Service Info endpoints (one per service) or include multiple service type declarations in a single response. The implementation approach depends on the server architecture and Service Info specification version. Implementers SHOULD clearly document how clients can discover all available services on a multi-service server. + +## Security considerations + +Service Info endpoints typically do not require authentication as they provide public metadata about service capabilities. However, implementers SHOULD consider: + +- **Rate limiting**: Prevent abuse of discovery endpoints +- **Information disclosure**: Avoid exposing sensitive internal details in service descriptions +- **Version information**: Consider security implications of advertising specific software versions +- **Registry integrity**: TASC MUST ensure the registry itself is protected from unauthorized modifications + +Implementers MAY choose to require authentication for Service Info endpoints in security-sensitive environments, but this may limit discoverability. + +## Use cases + +### Use case - service discovery in networks + +**Scenario**: A client needs to locate DRS (Data Repository Service) servers within a federated GA4GH network. + +**Solution**: The client queries known network nodes for their Service Info endpoints, filters responses by service type `{"group": "org.ga4gh", "artifact": "drs"}`, and identifies available DRS servers without manual configuration. + +**Benefits**: Automatic service discovery, reduced configuration burden, dynamic network adaptation. + +### Use case - capability negotiation + +**Scenario**: A workflow engine needs to determine if a server supports specific GA4GH APIs before submitting jobs. + +**Solution**: The engine queries the server's Service Info endpoint, checks for the required service types (e.g., TES, WES), and validates version compatibility before proceeding. + +**Benefits**: Prevents runtime failures, enables graceful degradation, supports multi-version environments. + +### Use case - monitoring and observability + +**Scenario**: A network operator needs to monitor the health and availability of GA4GH services across multiple institutions. + +**Solution**: Monitoring tools periodically query Service Info endpoints across the network, track service availability, detect version changes, and alert on inconsistencies or outages. + +**Benefits**: Proactive issue detection, network-wide visibility, compliance verification. + +## Considerations + +### Existing GA4GH products + +Existing GA4GH products that pre-date this recommendation MAY have already established service type identifiers outside the registry. These products SHOULD register their identifiers retrospectively to ensure completeness of the registry. Identifiers used in production deployments MUST be preserved to maintain backward compatibility, even if they don't fully conform to current naming conventions. TASC MAY grant exceptions for established identifiers on a case-by-case basis. + +### Non-GA4GH service types + +Organizations implementing GA4GH APIs alongside proprietary or non-standard services MAY use custom group identifiers outside the "org.ga4gh" namespace. The registry maintained by TASC is authoritative only for "org.ga4gh" service types. Organizations maintaining custom registries SHOULD follow similar governance practices to prevent confusion and identifier collisions within their own ecosystems. + +## References + +[SERVICE-INFO-SPEC] - GA4GH Service Info Specification: https://github.com/ga4gh-discovery/ga4gh-service-info +[TASC-REGISTRY] - GA4GH Service Info Type Registry: https://github.com/ga4gh/TASC/tree/main/service-info +[PRODUCT-APPROVAL] - GA4GH Product Approval Process: https://www.ga4gh.org/how-we-work/approval-process/ +[RFC-3986] - URI Generic Syntax: https://www.rfc-editor.org/rfc/rfc3986 +[KEBAB-CASE] - Naming Convention Best Practices: https://en.wikipedia.org/wiki/Letter_case#Kebab_case + +## Contributors + +Need to fill in here with all contributors and reviewers who have provided input. Contributions should be matched against [CRediT](https://credit.niso.org/). + +Mamana Mbiyavanga: Writing – original draft, Writing – review & editing +Technical Alignment Sub Committee (TASC): Investigation, Writing - review & editing diff --git a/recommendations/TASC Technical Specification Maintenance.md b/recommendations/TASC Technical Specification Maintenance.md new file mode 100644 index 0000000..cade7fd --- /dev/null +++ b/recommendations/TASC Technical Specification Maintenance.md @@ -0,0 +1,199 @@ +# TASC Technical Specification Development + +**Source**: TASC +**Recommendation**: GA4GH-REC-04 +**Title**: TASC Technical Specification Development: Policy and processes for developing and communicating maturity of GA4GH Technical Specifications +**Related GitHub issues**: [#64](https://github.com/ga4gh/TASC/issues/64), [#49](https://github.com/ga4gh/TASC/issues/49), [#16](https://github.com/ga4gh/TASC/issues/16), [#46](https://github.com/ga4gh/TASC/issues/46) +**Raised by**: Alex Wagner (GKS) +**Authors**: Alex Wagner, Larry Babb, Robert Freimuth, GKS Work Stream +**Date:** 2025-07-18 +**Status:** Approved +**Keywords**: maturity-model, versioning, specification, lifecycle +**Work Streams Impacted**: All work streams +**Products Affected**: All GA4GH specifications + +## Table of contents + +- [Abstract](#abstract) +- [Document Purpose](#document-purpose) +- [Alignment with the GA4GH Product Development Processes](#alignment-with-the-ga4gh-product-development-processes) +- [Feature Maturity Levels](#feature-maturity-levels) + - [Maturity Level Criteria](#maturity-level-criteria) + - [Maturity Advancement Process](#maturity-advancement-process) + - [Data Class Inheritance and Property Maturity](#data-class-inheritance-and-property-maturity) + - [Communicating Maturity Level](#communicating-maturity-level) +- [Product Feature Development Process](#product-feature-development-process) +- [Specification Releases and Versioning](#specification-releases-and-versioning) +- [Contributors](#contributors) + +## Abstract + +This policy defines the maturity model and release process for developing and maintaining GA4GH technical specifications. It establishes a four-level maturity framework (Draft, Trial Use, Normative, Deprecated) to communicate feature stability across the Innovation Adoption Lifecycle. The document describes advancement criteria, versioning rules following semantic versioning with maturity-based increments, annotation requirements in JSON Schema, and community ballot processes. This framework balances the need for specification evolution with stability commitments, enabling timely adoption while preventing breaking changes to mature features. Related Architectural Decision Records document the rationale for key design decisions. + +## **Document Purpose** + +The GA4GH is developing data exchange [**standards**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.1x756gyh0d77) for federated genomic data sharing. To address this, new [**technical specifications**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.hayv0rmdfa79) are required, such as the VRS standard, which must be developed and iterated upon through application across community [**implementations**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.xrzt0hhbuek1). This creates a tension between the need to create [**products**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.8r6pvv79hano) with enough stability for initial community adoption, while ensuring that they can evolve with minimal disruption to interoperate smoothly across a diverse set of genomic data resources. Mechanisms for communicating the stability, uptake, and development of technical specifications are therefore of paramount importance to addressing this balance. + +A maturity model is a useful mechanism for communicating varying stability across [**product features**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.ugrmhmrf3sz8) (e.g. [**data classes**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.i5ybdkaca3ah) or [**protocols**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.igglwc7cgyv6)) of a GA4GH standard. This is needed to help data producers at each stage of the adoption lifecycle (**Figure 1**) decide on the appropriate time to engage and implement the standard. Product features that have progressed through the maturity model should have an associated progression of support from the GA4GH specification maintainers for message generation, translation, and validation tooling. + +The purpose of this document is to clearly define the maturity model and release process for developing and maintaining GA4GH standards, with the goal of enabling timely specification adoption by the community. + +![][image1] + +***Figure 1 \- The Innovation Adoption Lifecycle*** ([**source**](https://en.wikipedia.org/wiki/Technology_adoption_life_cycle))***.** The Innovation Adoption Lifecycle illustrates adoption rates (y-axis) for new technologies over time (x-axis). Innovators (leftmost on the time axis) are among the first to adopt a new technology, and laggards (rightmost) are among the last, reflecting the differing needs for innovation and stability by these community groups. Adopters in every category along the innovation adoption lifecycle benefit from communication about the maturity of technical specification components generated in GA4GH technical products. Communicating when a component is ready for implementation by groups along the innovation / stability spectrum is a primary goal of the maturity model, enabling adopters to engage at a time that is appropriate for their organizational needs.* + +## *Alignment with the GA4GH Product Development Processes* + +Developing a new GA4GH standard would still require that the product go through the full GA4GH Product Development and Approval processes. This document complements those processes by providing a maturity, release, and versioning strategy that may be referenced by the product proposal for developing, maintaining, and extending standards using a maturity model for product features. The processes outlined here may also be applied to technical specifications that support downstream products but are not themselves GA4GH Standards (e.g. the [**GKS Common Library**](https://github.com/ga4gh/gks-common)). + +It is not expected that every category of product feature developed for a GA4GH technical specification will be annotated with the maturity model (e.g. validation tests or documentation appendices would likely not be annotated with maturity), though some categories (e.g. data classes and protocols) are expected to always be annotated with maturity levels. Generally, if a product feature would be sufficient for a major or minor version increment in the absence of a maturity system, it should be annotated with a maturity level using this system. + +# **Feature Maturity Levels** + +## *Maturity Level Criteria* + +| Level | Criteria | Specification Changes | Support | +| :---- | :---- | :---- | :---- | +| Draft | A requirements survey has been done, and a draft product feature has been developed and documented. | Not considered safe for stable implementation, changes may occur at any time and are published for evaluation by the implementation community. | Technical artifacts, documentation, and validation tests are produced and made publicly available. | +| Trial Use | The product feature has been reviewed by multiple data providers. It has been evaluated by the community and endorsed as ready for widespread testing. | Expected infrequently. Changes are made in consultation with all participants in the product group. | At least two reference implementations using the product feature, one of which must be open. | +| Normative | The product feature has been sufficiently tested to warrant long-term support from the specification maintainers. This is considered stable. | No backwards-incompatible changes will happen within the released major version. | The product feature will be fully supported by product reference implementations. | +| Deprecated | The product feature was previously released at a trial use or normative maturity, but will be discontinued in a subsequent version. | Use of the product feature will be discontinued in a subsequent minor version (for Trial Use maturity) or major version (for Normative maturity). | This product feature will no longer be supported once removed from the specification. | + +***Table 1 \- Product feature maturity level criteria and commitments.*** + +## *Maturity Advancement Process* + +Product feature maturity levels are to be reviewed and advanced by consensus among defined decision-makers following Work Stream and GA4GH processes, in consultation with the associated [**product group**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.ij15j31aj6jd) membership. Factors to be considered for product feature maturity advancement include the criteria specified in **Table 1**, the degree of adoption observed in the community, feedback provided by adopters, and availability of specification maintainers to provide the level of support required. + +### Developing a Draft Product Feature + +**Decision-makers: [Feature developers](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.69fvnn4tbegq)**, [**product owners**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.j3wp65n0i1d4) + +**Criteria:** *Draft* product feature development work should be based on real use cases across multiple environments (aligned with [**GA4GH Product Development 14.5**](https://www.ga4gh.org/our-products/development-and-approval-process/#section_5:~:text=14.5%20Development%20work%20should%20be%20based%20on%20real%20use%20cases%20across%20multiple%20environments.)). Requirements may result directly from a [**landscape analysis of the problem domain**](https://www.ga4gh.org/our-products/development-and-approval-process/#section_4), or may emerge in the course of technical specification development. It is expected that the need for product features are first discussed in a community forum (e.g. GitHub Discussions, product group calls). + +**Process:** Follow [**the GA4GH product feature development process**](#product-feature-development-process). As part of this process, it is expected that consensus among the decision-makers was reached and major design decisions documented. Disagreements are resolved per Work Stream and GA4GH processes. + +### Advancing from Draft to Trial Use + +**Decision-makers: [Feature developers](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.69fvnn4tbegq)**, [**product owners**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.j3wp65n0i1d4), [**product implementers**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.cg3l1hng99o2) + +**Criteria:** Advancing a *draft* product feature to *trial use* should include at least two independent product implementers that commit to supporting the *draft* product feature once it has been advanced to *trial use*. At least one of these implementations must be open (aligned with [**GA4GH Product Development 14.8.3**](https://www.ga4gh.org/our-products/development-and-approval-process/#section_5:~:text=14.8.3%20implementations)). Advancing a product feature to *trial use* also mandates a minor version increment at the next [**release**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.1ent9dk649ub). As part of this process, it is expected that consensus among the decision-makers was reached and major design decisions documented. Disagreement resolution is handled per Work Stream and GA4GH processes. + +**Process:** A [**ballot release**](#pre-releases) is created that describes draft models under evaluation for advancement to trial use. A survey is sent to all [**Product Implementers**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.cg3l1hng99o2) that have indicated they are implementing one or more features under evaluation for advance to Trial Use. This survey includes: + +1. Name of [**Product Implementer**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.cg3l1hng99o2) +2. Selection of a previously described implementation +3. If (or if multiple, which) product feature(s) are suitable for advance to Trial Use +4. Comments on response (e.g. explicit endorsement or description of gaps) + +There is a minimum 1-week review period for Product Implementers to respond, though this may be longer at the discretion of the product owners. More time for individual contributors may be permitted on request. + +### Advancing from Trial Use to Normative + +**Decision-makers: [Feature developers](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.69fvnn4tbegq)**, [**product owners**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.j3wp65n0i1d4), [**product implementers**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.cg3l1hng99o2), [**Work Stream leads**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.mg8fktfu4p72) + +**Criteria:** A *normative* model should have demonstrated interoperability of multiple data generation and data consumption implementations, and should include implementations beyond those used to advance a model to Trial Use. Advancing a product feature to *normative* also mandates a minor version increment at the next [**release**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.1ent9dk649ub). As part of this process, it is expected that consensus among the decision-makers was reached and major design decisions documented. Community consultation and disagreement resolution are handled per Work Stream and GA4GH processes. + +## *Data Class Inheritance and Property Maturity* + +[**Data models**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.1jbzkl8z59lm) may (and often do) include child [**data classes**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.i5ybdkaca3ah) that inherit properties from a parent data class. For example, the [**Entity**](https://github.com/ga4gh/gks-common/blob/1b7e52d6013c6785300aa933efa9210e2aafa57b/schema/gks.common-source.yaml#L7-L31) data class from the GKS Common Library provides shared properties (e.g. *id*, *label*, *extensions*) that are inherited by several child data classes across the VRS and Variant Annotation specifications. + +To address inheritance used in a data model, we place additional constraints on the maturity of data classes and their properties. First, child data classes may not have maturity levels greater than the upstream data classes they inherit from. Second, the properties of a data class may not exceed the maturity of the data class as a whole. Together, these rules ensure that attention to the maturity of upstream classes is addressed first, and that less mature data classes do not artificially convey stability (in whole or in part) through inheritance of properties of more mature data classes. + +These rules also allow for extending more mature data classes with new properties that exist in a less mature state. + +## *Communicating Maturity Level* + +Minimally, primary documentation sites (e.g. [**vrs.ga4gh.org**](https://vrs.ga4gh.org/)) will annotate data classes, data class properties, protocols, and other important documentation with their corresponding maturity levels. + +In JSON Schema, this is accomplished using the maturity property for data classes (see the JSON Schema [**maturity annotation for the VRS Allele**](https://github.com/ga4gh/vrs/blob/454c5312e8e425eb170901c7520311f3ca7904e3/schema/vrs/json/Allele#L6) class) or data class properties (see this JSON Schema [**property-level maturity annotation in the VA-Spec Cohort Allele Frequency**](https://github.com/ga4gh/va-spec/blob/4c14e9f7f033dce3b6701ecd0fccca415476fd76/schema/va-spec/profiles/caf/json/CohortAlleleFrequency#L142-L143) profile). + +# **Product Feature Development Process** {#product-feature-development-process} + +Development of GA4GH standards involves a community-oriented process that iterates on the general pattern of: + +1. Discuss Issues +2. Gather Requirements +3. Propose Solutions +4. Develop Product Feature + +## *Discuss Issues* + +Emerging discussion topics should first be created as Discussions in the repository of the associated GA4GH product (e.g. the VRS Discussion board at [**github.com/ga4gh/vrs/discussions**](https://github.com/ga4gh/vrs/discussions)). The [**product owners**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.j3wp65n0i1d4) monitor these discussions and coordinate their addition to the agenda on community calls. + +## *Gather Requirements* + +On a community call, the discussion topic is first announced as a future call agendum by the [**product owners**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.j3wp65n0i1d4), and a request made for asynchronous discussion and community-driven requirements gathering on the GitHub Discussion thread. + +As part of requirements gathering, the community is surveyed for interest in implementing the feature, and any potential product feature implementers are expected to provide: + +1. Name of [**Product Implementer**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.cg3l1hng99o2) +2. A description of their [**Implementation**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.xrzt0hhbuek1): + 1. A short description of the implementation that will use / is using the standard + 2. If the implementation is maintained by a Driver Project + 1. If so, which? + 3. If the implementation is open / public +3. If (or if multiple, which) product feature(s) will be used by the implementation +4. If the implementer will contribute [**product feature developer**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.69fvnn4tbegq) effort + +An example form for collecting this information is the [**GKS Product Implementer Form**](https://docs.google.com/forms/d/e/1FAIpQLSfVKA6LmeDNYxH7ssnyk0ifRtLCgQKlZfoUzXxzO-h6JkX0og/viewform?usp=sf_link). + +It is expected that there will be at least two product implementers supporting a product feature, and that at least one implementation will be open. It is also expected that at least one product implementer will contribute product feature developer effort. + +Once the above criteria are met, the development process may advance. + +## *Propose Solutions* + +Product implementers should propose solutions on the GitHub Discussion thread. On a subsequent call, the topic is raised for review of requirements and discussion of proposed solutions. Action items may include advancing to solution implementation, furthering investigation of requirements, or continued discussion on a subsequent call. When one or more solutions are identified as ready to advance to implementation, a GitHub Issue is created (e.g. the VRS Issue board at [**github.com/ga4gh/vrs/issues**](https://github.com/ga4gh/vrs/discussions)) and assigned to one or more [**feature developers**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.69fvnn4tbegq). + +## *Develop Product Feature* + +The assigned [**feature developers**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.69fvnn4tbegq) will develop the product feature on a separate feature branch reflecting the associated GitHub Issue, and make a Pull Request for community review. The [**product owners**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.j3wp65n0i1d4) are responsible for review and recommend action on Pull Requests within 2 weeks. Once merged, the product feature is developed. + +# **Specification Releases and Versioning** + +## *Versioning* + +[**Versions**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.sz593b8vre9l) are used to identify releases of technical specifications, *not* to individual product features. + +Technical specification development is intrinsically linked to policy surrounding major and minor version identification, which follow semantic versioning v2 (SemVer; [**semver.org**](https://semver.org/#semantic-versioning-200)) practices for API versioning. Version syntax follows SemVer syntax. Examples of how product features at different maturity levels are applied to the SemVer major/minor/patch syntax as follows: + +### Major Version Increment + +* Backwards-incompatible changes to a *normative* product feature +* Backwards-incompatible changes to property names of a previously-released *normative* data class +* Backwards-incompatible changes to the definition of a previously-released *normative* data class +* Backwards-incompatible changes to the digests of previously-released *normative* data class (as applicable) +* Addition of required fields to previously-released *normative* data class + +### Minor Version Increment + +* Backwards-incompatible changes to a *trial use* product feature +* Addition of optional fields to data models at the *trial use* or *normative* level +* Release of a new product feature at the *trial use* or *normative* level +* Backwards-incompatible changes to property names of a previously-released *trial use* data class +* Backwards-incompatible changes to the definition of a previously-released *trial use* data class +* Backwards-incompatible changes to the digests of previously-released *trial use* data class (as applicable) +* Addition of required fields to previously-released *trial use* data class + +### Patch Version Increment + +* A new product feature at the *draft* maturity level +* Any changes made to *draft* product features +* Addition of implementation guidance, tests, or other supporting product features that do not directly affect data compatibility + +Versioning of approved GA4GH standards should additionally follow the procedures for [**GA4GH Product Updates**](https://www.ga4gh.org/our-products/development-and-approval-process/#section_7). Specifically, advancement of data classes to the *trial use* or *normative* levels must be accompanied by a minor release increment, and therefore may only be included in a release following an appropriate community and PRC consultation process ([**GA4GH Product Development 32**](https://www.ga4gh.org/our-products/development-and-approval-process/#section_7:~:text=32.%20Public%20comment,reduced%20or%20omitted.)). + +## *Releases* + +A [**release**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.1ent9dk649ub) of a technical specification contains all of the content of the specification repository. This includes all features, including [**data models**](https://docs.google.com/document/d/1xPFXRF7_Ppe5SDBHTBa1E-MBHHNtoc1Q-jZ1olOrtc4/edit?tab=t.0#heading=h.1jbzkl8z59lm) (source and derived artifacts), linked upstream dependencies, documentation, implementation guidance, validation tests, and examples. Releases provide a comprehensive and static snapshot of a technical specification that may be referenced for adoption by downstream products and implementations. + +### Pre-releases {#pre-releases} + +In order to support continuous development of a technical specification, pre-release snapshots are allowed and must use the SemVer syntax for pre-releases. Pre-release snapshots may be created for purpose at any time by the product leads. Examples of pre-release snapshots following this process may be found [in the VRS repository](https://github.com/ga4gh/vrs/releases). + +[image1]: + +## Contributors + +- GKS Work Stream diff --git a/recommendations/template.md b/recommendations/template.md new file mode 100644 index 0000000..29a0b2c --- /dev/null +++ b/recommendations/template.md @@ -0,0 +1,73 @@ +# [Short Recommendation Title] + +**Source**: TASC +**Recommendation**: GA4GH-REC-XX +**Title**: [Full descriptive title] +**Related GitHub issues**: [#N](https://github.com/ga4gh/TASC/issues/N) +**Raised by**: [Name (Work Stream), e.g., Alex Wagner (GKS), Mamana Mbiyavanga (LSG)] +**Authors**: [Author names, comma-separated] +**Date:** YYYY-MM-DD +**Status:** [Draft | Approved | Superseded] +**Keywords**: [e.g., api, pagination, interoperability, security, governance, versioning] +**Work Streams Impacted**: [e.g., Cloud, Discovery, DaMaSC] +**Products Affected**: [e.g., Beacon v2, TRS, Refget] + +## Abstract + +[A concise summary of the recommendation: what it covers, why it exists, and its scope. Typically one paragraph that gives the reader enough context to decide whether the full document is relevant to them.] + +## Table of contents + +- [Recommendation](#recommendation) +- [Background](#background) +- [Detailed Guidance](#detailed-guidance) +- [Use Cases](#use-cases) +- [Considerations](#considerations) +- [References](#references) +- [Contributors](#contributors) + +## Recommendation + +[The normative recommendation statement. Use RFC 2119 keywords (MUST, SHOULD, MAY) to express requirements. This section should be self-contained — a reader should understand what to do from this section alone. State what is required, what is optional, and who it applies to.] + +## Background + +[Context and motivation for the recommendation. Why does this problem matter? What existing practices or gaps led to this recommendation? Reference relevant GA4GH work streams, prior discussions, or industry standards.] + +## Detailed Guidance + +[The main body of the recommendation. Break into subsections as appropriate for the topic. Include technical details, implementation patterns, examples, and diagrams where helpful. This section will vary significantly between recommendations — adapt the structure to fit the content.] + +### [Subsection Title] + +[Subsection content] + +### [Subsection Title] + +[Subsection content] + +## Use Cases + +[Concrete scenarios illustrating when and how this recommendation applies. Help readers understand the practical impact.] + +### Use case - [descriptive title] + +[Description of the scenario and how the recommendation applies] + +## Considerations + +[Important caveats, edge cases, backward compatibility notes, or known limitations. Include any impact on existing GA4GH products.] + +## References + +[Cited standards, specifications, external resources, and related GA4GH documents. Use descriptive labels for link references.] + +- [LABEL] [Description and link] + +## Contributors + +[People who contributed to the development of this recommendation, including reviewers and discussion participants.] + +| Name | Organisation | +|------|-------------| +| [Name] | [Organisation] |