Check teams from branch PRs

[jaimergp]

Description

• Follow up to Documenting sub-teams as YAML files (and verified using a JSON Schema) #147
• Closes Record-keeping for subteams #86

[jaimergp]

Ok, I think we are getting closer to something functional here.

Summary of features:

• scripts/check_teams.py will check whether the local files match Github's info:
  • Are all teams mentioned in a file?
  • Do memberships match?
  • Does repo access match?
  • Are there any users with individual access?
  • Do descriptions match?
• scripts/schemas.py encodes the YAML document schema. It also has a generate argument that will dump the Github info to a file, if missing. Useful for bootstrapping new teams.
• steering.csv and emeritus.csv are also expressed as YAML now.
• This is paired with Republish governance, code of conduct and teams from conda/governance conda-dot-org#285 for pretty printing.

Pending work:

• I combed through the organizations and issues to add as many links for membership provenance as possible, but a few are still missing.
• Some teams are duplicated across orgs. Some in conda-incubator don't make sense anymore, we should remove them?
• Team descriptions probably need some work. And we should probably add a long_description field too so teams can add more docs about how they function, responsibilities, etc.
• Individual access checks need a more powerful token, I think. Or maybe we don't want to check this in all repositories, only key ones.

I know this is going to be a nightmare to review, so I suggest merging a good baseline we are happy with, and adjusting the files in several PRs little by little until the CI passes. Especially if we need to update team composition and other metadata in Github (for bookkeeping).

[danyeaw]

This is amazing @jaimergp! I agree that getting this merged as a baseline that then we can update overtime is a great strategy.

[jaimergp]

Posting the pre-commit logs here because they don't fit in pre-commit.ci:

Validate files with jsonschema...........................................Failed
- hook id: check-jsonschema
- exit code: 1
  Schema validation errors were encountered.
    teams/conda/community-contributors.yml::$.description: None is not of type 'string'
    teams/conda/community-contributors.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda/community-contributors.yml::$.charter: None is not of type 'string'
    teams/conda/community-contributors.yml::$.links: None is not of type 'array'
    teams/conda-incubator/ceps.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/ceps.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/ceps.yml::$.links: None is not of type 'array'
    teams/conda-incubator/setup-miniconda.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/setup-miniconda.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/setup-miniconda.yml::$.links: None is not of type 'array'
  Schema validation errors were encountered.
    teams/conda-incubator/constructor.yml::$.description: None is not of type 'string'
    teams/conda-incubator/constructor.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/constructor.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/bots.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/bots.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/bots.yml::$.links: None is not of type 'array'
  Schema validation errors were encountered.
    teams/conda-incubator/rattler.yml::$.description: None is not of type 'string'
    teams/conda-incubator/rattler.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/rattler.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/rattler.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-project.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-project.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/conda-project.yml::$.links: None is not of type 'array'
    teams/conda-incubator/infrastructure.yml::$.description: None is not of type 'string'
    teams/conda-incubator/infrastructure.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/infrastructure.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/infrastructure.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-mirror.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-mirror.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/conda-mirror.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-sh.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-sh.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/conda-sh.yml::$.links: None is not of type 'array'
    teams/conda-incubator/shells.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/shells.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/shells.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-lockfiles.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-lockfiles.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/conda-lockfiles.yml::$.links: None is not of type 'array'
  Schema validation errors were encountered.
    teams/conda/conda-maintainers-project-management.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-colab.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-colab.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/conda-colab.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-oci.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-oci.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/conda-oci.yml::$.links: None is not of type 'array'
    teams/conda/ceps.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda/ceps.yml::$.charter: None is not of type 'string'
    teams/conda/ceps.yml::$.links: None is not of type 'array'
  Schema validation errors were encountered.
    teams/conda/conda-core.yml::$.description: None is not of type 'string'
    teams/conda/conda-core.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda/conda-core.yml::$.charter: None is not of type 'string'
    teams/conda/conda-core.yml::$.links: None is not of type 'array'
    teams/conda-incubator/builds-tools.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/builds-tools.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/hatch-conda-build.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/hatch-conda-build.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/hatch-conda-build.yml::$.links: None is not of type 'array'
    teams/conda-incubator/condastats.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/condastats.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/condastats.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-dot-org.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-dot-org.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/conda-dot-org.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-completions.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-completions.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/conda-completions.yml::$.links: None is not of type 'array'
  Schema validation errors were encountered.
    teams/conda-incubator/code-of-conduct.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/code-of-conduct.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/code-of-conduct.yml::$.links: None is not of type 'array'
    teams/conda/distribution-contractors.yml::$.description: None is not of type 'string'
    teams/conda/distribution-contractors.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda/distribution-contractors.yml::$.charter: None is not of type 'string'
    teams/conda/distribution-contractors.yml::$.links: None is not of type 'array'
    teams/conda-incubator/design.yml::$.description: None is not of type 'string'
    teams/conda-incubator/design.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/design.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/design.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-tutorial.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-tutorial.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/conda-tutorial.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-env-builder.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-env-builder.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/conda-env-builder.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-libmamba-solver.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-libmamba-solver.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/installer.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/installer.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/installer.yml::$.links: None is not of type 'array'
  Schema validation errors were encountered.
    teams/conda-incubator/docs.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/docs.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/docs.yml::$.links: None is not of type 'array'
    teams/conda-incubator/implementation.yml::$.description: None is not of type 'string'
    teams/conda-incubator/implementation.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/implementation.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/implementation.yml::$.links: None is not of type 'array'
    teams/conda-incubator/steering-council.yml::$.description: None is not of type 'string'
    teams/conda-incubator/steering-council.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/steering-council.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/steering-council.yml::$.links: None is not of type 'array'
    teams/conda-incubator/conda-plugins.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/conda-plugins.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/conda-plugins.yml::$.links: None is not of type 'array'
    teams/conda-incubator/fetch.yml::$.description: None is not of type 'string'
    teams/conda-incubator/fetch.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/fetch.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/fetch.yml::$.links: None is not of type 'array'
  Schema validation errors were encountered.
    teams/conda-incubator/content.yml::$.description: None is not of type 'string'
    teams/conda-incubator/content.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/content.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/content.yml::$.links: None is not of type 'array'
    teams/conda-incubator/relock-conda.yml::$.charter: None is not one of ['dynamic', 'static', 'project']
    teams/conda-incubator/relock-conda.yml::$.charter: None is not of type 'string'
    teams/conda-incubator/relock-conda.yml::$.links: None is not of type 'array'

[jezdez]

I don't agree with merging snapshots of the progress here as a baseline, since that has the smell of not taking governance information seriously, and I don't want to end up in legal debates later about why we merged this.

That said, if you need a break from this, or frankly if you don't have the access needed because of the org ownership situation, I'm happy to devote time to carry this over the finish line with the items you wrote above. I'm happy to devote time to it this week.

[jaimergp]

That said, if you need a break from this, or frankly if you don't have the access needed because of the org ownership situation, I'm happy to devote time to carry this over the finish line with the items you wrote above. I'm happy to devote time to it this week.

I have time and energy for this, but I don't have permissions to do any cleanups in the conda organization. I can do the ones in conda-incubator, but even for that I'd like consensus.

I don't agree with merging snapshots of the progress here as a baseline, since that has the smell of not taking governance information seriously, and I don't want to end up in legal debates later about why we merged this.

I don't have strong opinions on how we proceed, but some counterpoints:

• main already contains in-progress work.
• the situation is quite dire with outdated permissions.
• it will be very tricky to follow cleanup decisions if all the teams are removed as part of the same PR. I'd prefer having one team (or a few related ones) removed per PR for traceability.

[jaimergp]

Oh, and I also need help reviewing the types of teams. Not clear how some of them were created and with which mission.

[jaimergp]

• @kenodegard shared that a governance-defined team does not necessarily correspond to a Github team
• Hence, a YAML file should be able to represent the same team across several organizations
• The schema needs to be updated to reflect that, and then we can do the different directories for conda and conda-incubator. To extend this to arbitrary organizations and repositories, we'll probably need to enumerate in which organizations the team must be found with given members. And also which repositories should they have access too (in each organization, but that's trivial to match).

[jezdez]

FYI, I just renamed the builds tools team to build tools team, as I think this was a typo when we created the team.

[jaimergp]

We will also need to rename the Zulip channel: #builds-tools