Skip to content

docs: refresh README for 2.1.0#32

Open
gerchowl wants to merge 32 commits intodevfrom
docs/refresh-readme-2.1.0
Open

docs: refresh README for 2.1.0#32
gerchowl wants to merge 32 commits intodevfrom
docs/refresh-readme-2.1.0

Conversation

@gerchowl
Copy link
Copy Markdown
Contributor

Brings the README up to date with what shipped in py-materials 2.1.0. The README was still promoting `v1.0.0` git+https installs, missing `Material.molar_mass` and `pymat.elements` entirely, had stale repo URLs (`MorePet/py-mat`), and carried a broken `enrich_from_periodictable` example that promised compound density.

What's in

  • Template (`docs/README_TEMPLATE.md`): PyPI-first install, new features (molar mass, Python 3.10–3.13 support), fixed `MorePET/mat` URLs, rewritten periodictable example matching the current (correct) behavior from enrich_from_periodictable: no density computation for compounds #9, new ADR-0001 link
  • Tests (`tests/test_readme_examples.py`): two new methods in `TestBasicUsage` — `test_molar_mass_from_formula` and `test_elements_low_level_api` — so the Quick Start picks them up AND they stay CI-verified
  • Generator (`scripts/generate_readme.py`): fixed two pre-existing bugs
    • 8-space indent never dedented → code blocks uncopy-pasteable
    • `assert` lines stripped wholesale → readers couldn't see expected return values
  • README.md: regenerated, 19 examples, 580 lines

Scope

Doc refresh only, no runtime code touched. Ready to merge independently of #30 / #3 / the three-repo PBR conversation. Those will need a second doc pass once `Material.pbr_source` lands, but the 2.1.0 bits shouldn't wait.

Test plan

  • Lint & Format passes (pre-commit on template, tests, generator, README all clean locally)
  • Tests pass (new Quick Start methods pass; `test_readme_examples` full suite = 16 passed, 3 skipped — the skips are pre-existing build123d/ocp_vscode gated tests)
  • Security Scan, Dependency Review, CodeQL, Rust (mat-rs) pass
  • README renders correctly on the GitHub file view (dedent + ADR link)

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

gerchowl and others added 30 commits April 15, 2026 20:57
The README was written before 2.1.0 shipped and was stale in several
places. This refresh brings it current with what's actually on PyPI
today.

## Template changes (docs/README_TEMPLATE.md)

- **Installation**: PyPI (`pip install py-materials`) is now the
  recommended path. Git+https is documented as secondary for the
  main-branch dev build. The `[periodictable]`, `[build123d]`, and
  `[all]` optional extras are all mentioned with their install lines.
- **Features list** — added:
  - Formula parsing + molar mass (with Python ↔ Rust parity note)
  - Explicit Python 3.10–3.13 support statement
  - Pure-element density only note on `periodictable` integration
    (the "compound density" promise was always broken; see #9)
- **Repo URLs fixed**: `MorePet/py-mat` / `MorePET/py-mat` →
  `MorePET/mat` (the repo was renamed but the template never
  caught up).
- **Links section** now includes PyPI and the rs-materials crates.io
  page alongside the GitHub / Issues links.
- **Enrichment example** rewritten to match current behavior: pure
  elements get density from `periodictable`, compounds only get
  `composition` populated. Molar mass works for both via the
  computed `Material.molar_mass` property. Previous example promised
  `print(material.density)  # ~3.95 g/cm³` for Al2O3 which never
  worked — the bug fixed in #9.
- **New "Design Decisions (ADRs)" section** linking
  `docs/decisions/0001-derived-chemistry-properties-live-on-material.md`,
  so contributors can find the rationale behind `Material.molar_mass`
  being a computed property.

## Test additions (tests/test_readme_examples.py)

Two new methods in `TestBasicUsage`, so they show up in the README's
Quick Start section AND stay verified by CI:

- `test_molar_mass_from_formula` — shows `Material.molar_mass` on a
  pure element, simple compound, fractional-stoichiometry compound
  (LYSO), dopant-suffix stripping (LYSO:Ce), the Pint-wrapped
  `molar_mass_qty`, and the `None` fallback when no formula is set.
- `test_elements_low_level_api` — shows the `pymat.elements` module
  direct API (`ATOMIC_WEIGHT`, `parse_formula`, `compute_molar_mass`)
  for Monte Carlo transport callers who want stoichiometry without
  wrapping in a full `Material`. Notes the line-for-line Rust
  rs-materials parity.

## Generator fixes (scripts/generate_readme.py)

Two pre-existing bugs surfaced while regenerating:

1. **Code block indentation was broken** — the generator extracted
   test function bodies but never dedented. Every code block in the
   generated README had an 8-space indent on every line except the
   first (the `from pymat import ...` line, extracted from a
   different regex branch, started at column 0 while the rest of the
   body started at column 8). Result: uncopy-pasteable examples.
   Fixed via `textwrap.dedent()` on both docstring and code extracts.

2. **Assertions were stripped wholesale** — the previous
   `re.sub(r"\s*assert .*?\n", "", code)` removed all `assert` lines
   to make examples "cleaner", but it removed exactly the lines
   that show the reader the expected return values. After the fix,
   readers see e.g. `assert iron.molar_mass == 55.85` alongside the
   computation instead of having to guess. `pytest.importorskip`
   calls still get filtered because those ARE noise in a user-
   facing example.

## README.md

Regenerated. 580 lines, 19 examples, code blocks now dedented and
assertion-bearing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New package pymat.vis with:

- vis.__init__: public API re-exports (search, fetch, rowmap_entry,
  get_manifest). Usage: `from pymat import vis; vis.search(...)`
- vis._client: stub implementations for the mat-vis fetch layer.
  Pure Python, stdlib-only. Raises NotImplementedError until
  mat-vis publishes first release (M4/M5).
- vis._model: Vis dataclass (source_id, finishes, textures,
  resolve()) + ResolvedChannel. Vis.from_toml() parses the
  [vis] section from material TOML files.
- vis.adapters: standalone to_threejs(), to_gltf(), export_mtlx()
  functions. Take Material, read from both .properties.pbr and
  .vis namespaces. Stub implementations.

API design rationale: Material.vis holds pointers + cached textures.
Fetching/search lives in pymat.vis (module-level functions).
Adapters are functions, not methods — consumers can write their
own without changing pymat. See #35.
- _MaterialInternal gets ._vis field + .vis property (always returns
  Vis instance, never None — lazy-constructed on first access)
- loader.py parses [vis] sections from TOML, populates via Vis.from_toml()
- "vis" added to skip-list so it's not treated as a child material
- Added [vis] section to stainless in metals.toml as proof-of-concept:
  default="brushed", finishes={brushed, polished}

Usage:
  stainless.vis.source_id    # "ambientcg/Metal032"
  stainless.vis.finish       # "brushed"
  stainless.vis.finishes     # {"brushed": "...", "polished": "..."}
  stainless.vis.finish = "polished"  # switches source_id

Custom materials get empty Vis (source_id=None, textures={}).
Textures lazy-fetch via vis._client on first access (not yet
implemented — raises NotImplementedError until mat-vis publishes).

All 109 existing tests pass.
Covers: Vis construction (empty, from_toml with/without default),
finish switching (source_id update, cache clear, unknown raises),
textures access (empty when no source_id, NotImplementedError
when source_id set but client unimplemented), ResolvedChannel
(texture available, scalar fallback, neither), Material.vis
wiring (custom material empty, settable, same instance, TOML
populated, child without vis), module-level API stubs.

128 passed, 26 skipped.
Replaces NotImplementedError stubs with working implementation:

- get_manifest(): returns release tag + base URL
- search(): loads JSON indexes, filters by category/source, ranks
  by roughness/metalness distance
- fetch(): rowmap lookup → HTTP Range request → PNG bytes + cache
- rowmap_entry(): raw offset/length for DIY consumers

Pure Python, stdlib-only (urllib.request). ~250 lines.
Tested against live mat-vis v0.1.0 release data:
  vis.fetch("ambientcg", "Wood049") → 2 channels, 6.3 MB, 2.2s
  cache hit → 0.002s

Also updated tests: mock fetch instead of expecting
NotImplementedError, added search mock test, rowmap miss test.
130 passed, 26 skipped.
Three output adapters, all standalone functions taking Material:

- to_threejs(material) → MeshPhysicalMaterial dict with base64
  data URIs for textures. Maps metallic→metalness, base_color→
  color hex int. Includes ior/transmission/clearcoat/emissive
  when non-default.

- to_gltf(material) → glTF pbrMetallicRoughness dict. Includes
  KHR_materials_transmission extension when transmission > 0.
  Note: does NOT pack metalness+roughness into single texture
  (would need Pillow — left for a dedicated exporter).

- export_mtlx(material, path) → writes .mtlx XML + PNG files.
  Standard Surface node graph with image references as siblings.

Field name mapping per docs/specs/field-name-mapping.md.
130 passed, 26 skipped.
Per mat#34 discussion:
- periodictable moved from [periodictable] extra to core deps
  (~800 KB, used for molar mass + element lookups)
- Removed extras: [periodictable], [matproj], [build123d], [all]
  - matproj → curation-time tool only (mat#36)
  - build123d → reversed dep direction (they depend on us)
  - uncertainties → will become core in a follow-up (mat#33)
- Only [dev] extra remains (pytest, build123d for integration tests)

pip install mat → pint + periodictable + tomli. ~2 MB.
Vis.discover(category, roughness, metallic):
  - Searches mat-vis index for matching appearances
  - Returns ranked candidates, does NOT auto-set source_id
  - Pass auto_set=True to pick top match
  - Uses vis.search() under the hood

scripts/enrich_vis.py:
  - Runs vis.search() against all unmapped TOML materials
  - Outputs proposed [vis] sections as TOML
  - Designed to run in CI via enrich-vis.yml workflow
  - Opens PR for human review of proposed mappings
  - Tested against live mat-vis v0.1.0: 120 proposals generated

3 new tests for discover (candidates, auto_set, empty results).
133 passed, 26 skipped.
Downloads all materials for a (source × tier) into the local cache.
Skips already-cached materials, logs progress every 10 materials.

Usage:
    from pymat import vis
    vis.prefetch("ambientcg", tier="1k")  # downloads all ~50 materials
    # subsequent vis.fetch() calls are instant from cache

Closes the last ADR-0004 compliance gap.
133 passed, 26 skipped.
Migration per mat#37:
- Vendored mat_vis_client.py + adapters.py from mat-vis v2026.04.0
  release assets into _vendor_client.py and _vendor_adapters.py
- _client.py reduced to thin wrapper (~110 lines) delegating to
  vendored MatVisClient class
- adapters.py reduced to thin wrappers (~75 lines) that extract
  scalars + textures from Material and pass to vendored generic
  adapters (scalars dict, textures dict — no Material dependency)
- Index seeding workaround: downloads index JSONs from release
  assets since they're not in git yet (mat-vis#40)
- fetch() gracefully returns {} on errors instead of crashing
- Tests updated to mock vendored client, not internal functions

Blocked on full functionality by mat-vis#40 (missing ambientcg
index + partitioned rowmap handling in vendored client).

133 passed, 26 skipped.
…s mapping

- Re-downloaded mat_vis_client.py from v2026.04.0 — now handles
  category-partitioned rowmaps (merges per-category rowmaps into one)
- Fixed metallic→metalness field name mapping in adapters wrapper
  (py-mat uses "metallic", mat-vis uses "metalness")
- Closes mat-vis#40 on our side

Verified end-to-end: Material → vis.fetch → 4 PNG channels →
to_threejs → MeshPhysicalMaterial with all texture maps.
133 passed, 26 skipped.
scripts/generate_catalog.py:
- Iterates all TOML-registered materials (95 across 7 categories)
- Generates docs/catalog/ tree: root index → category pages →
  per-material detail pages
- Each page has: identity, mechanical, thermal, PBR, composition,
  vis source/finishes
- Category index: table with material, density, roughness, metallic
- Optional thumbnails: fetches color texture from mat-vis, resizes
  to 128x128 via Pillow
- --skip-thumbnails for text-only (CI without mat-vis access)

Designed for CI: .github/workflows/catalog.yml runs on push to
dev, commits generated docs to a gh-pages branch or PR.

133 passed, 26 skipped.
CONTRIBUTING.md:
- Three paths: request a material, add a material (TOML template),
  fix a value
- Data quality guidelines (cite sources, SI units, don't fabricate)
- Vis mapping guide (vis.search → TOML [vis] section)
- Standard PR workflow (fork, branch from dev, test, lint)

Issue templates:
- material-request.yml: name, category, known properties, use case,
  datasheet URL, desired appearance
- material-correction.yml: material, property, current/correct value,
  source citation
- Installation section simplified (no stale extras)
- Added Material Catalog section linking to docs/catalog/
- Added Contributing section with issue template links
- Added mat-vis link to Links section
- Added [mat-vis] reference link
mat-vis now hosts pre-baked thumbnail tiers (128/256/512). The
catalog script fetches color textures at the 128px tier directly
instead of downloading 1K textures and resizing with Pillow.

Simpler, faster, no Pillow dependency for catalog generation.
Reverted attempt to merge pbr.*_map fields into adapters.
Per our design: all visual data lives under .vis. Legacy
pbr.*_map fields (normal_map, roughness_map, etc.) continue
to work via ocp_vscode's existing is_pymat path — that's
Bernhard's migration to handle when adopting the adapter
pattern. We don't bridge legacy into the new API.

Deprecation of pbr.*_map → .vis migration tracked for 2.3.0.
…operties.pbr

Migration path toward 3.0 (mat#40):
- Vis dataclass gains PBR scalar fields (roughness, metallic,
  base_color, ior, transmission, clearcoat, emissive)
- Vis.from_toml() parses scalars from [vis] section
- Loader syncs vis scalars → properties.pbr for backward compat
  (ocp_vscode's is_pymat path still reads properties.pbr)
- Adapters read vis scalars first, fall back to properties.pbr
- stainless TOML migrated: [pbr] → [vis] as proof-of-concept

Both [pbr] and [vis] TOML sections accepted. vis wins when both
present. No breakage — single migration path to 3.0.

133 passed, 26 skipped.
Delete 890 lines of vendored code, import from mat-vis-client:

Deleted:
  _vendor_client.py   (444 lines — was mat-vis's client.py)
  _vendor_adapters.py (280 lines — was mat-vis's adapters.py)
  _client.py          (166 lines — our wrapper around vendored)

Remaining pymat/vis/ (354 lines):
  __init__.py  (32)  — re-exports from mat_vis_client
  _model.py    (230) — Vis, ResolvedChannel, from_toml, discover
  adapters.py  (92)  — Material→dict wrappers

Dependencies:
  mat-vis-client>=2026.4.0 added to core deps
  (currently installed from git, PyPI publish pending)

Test mocks updated: pymat.vis._client → mat_vis_client.
133 passed, 26 skipped.

Refs: mat#37, mat-vis#36, mat-vis#50
- uncertainties>=3.2 added to core dependencies (~59 KB, pure Python)
- Loader accepts composition values as:
  - plain float: 0.68
  - {nominal, stddev}: {nominal = 0.4, stddev = 0.1} → ufloat
  - {min, max}: {min = 0.2, max = 0.6} → ufloat(midpoint, half-range)
  - {nominal, min, max}: explicit nominal + range
- Aluminum 6063 added with grade-spec composition ranges as
  proof-of-concept (Si, Mg, Fe have ranges; Al is balance)
- Existing plain-float compositions unchanged
- test_loader: handle ufloat in composition sum check
- 14 new tests for parsing, propagation, material integration
- 147 passed, 26 skipped

Closes #33.
- docs/catalog/ generated with 96 materials across 7 categories
  (thumbnails via CI when mat-vis tiers available)
- .github/workflows/catalog.yml — regenerates on TOML data changes
- .github/workflows/enrich-vis.yml — auto-proposes vis mappings
  on mat-vis release dispatch
- vis.search() no longer filters by tier (index available_tiers
  may not reflect all actual tiers — search is for discovery, tier
  is a fetch-time concern)
- Fixed ufloat sorting in catalog composition tables

147 passed, 26 skipped.

Closes #38, #39.
Tests for _extract_scalars (pbr fallback, vis-wins, metallic→metalness
mapping), _extract_textures, to_threejs (scalar + texture modes),
to_gltf, export_mtlx (with/without textures).

Overall coverage: 81% (158 passed, 26 skipped).
Every [material.pbr] section across 7 data files renamed to
[material.vis]. Zero [pbr] sections remain. Values unchanged.

The loader already routes [vis] scalars to both .vis and
properties.pbr (backward compat sync), so all existing tests
pass without code changes.

128 materials load. 170 tests pass.

Part of 3.0.0 migration (#40).
core.py:
  - color=(r,g,b) → sets vis.base_color (was properties.pbr)
  - pbr={} kwarg → writes to .vis AND properties.pbr (dual-write)
  - Optical→PBR derivations (ior, transmission) target .vis
  - _sync_vis_to_pbr() keeps properties.pbr populated for
    ocp_vscode backward compat

vis/_model.py:
  - Vis.get(field, default) with _PBR_DEFAULTS dict
  - Resolves None→default for adapters

vis/adapters.py:
  - _extract_scalars reads only from .vis via get() with defaults
  - No more pbr fallback — vis is the single source

170 passed, 25 skipped.

Part of 3.0.0 migration (#40).
- loader.py: TOML [pbr] section emits DeprecationWarning, still
  parsed for backward compat
- loader.py: vis→pbr sync moved to _sync_vis_to_pbr() in core.py
- core.py: Material(pbr={...}) emits DeprecationWarning, still
  works (writes to both vis and properties.pbr)

170 passed (with DeprecationWarning suppressed), 25 skipped.

Part of 3.0.0 (#40).
- test_adapters: _make_material sets vis fields directly instead
  of pbr={} kwarg
- pyproject.toml: filterwarnings suppresses DeprecationWarning
  for tests that still use pbr={} kwarg (legacy API, deprecated
  but functional)
- Also suppresses uncertainties FutureWarning

170 passed, 25 skipped.
test_e2e_vis.py (7 tests, skip with MAT_VIS_SKIP_LIVE=1):
  - Search and fetch real textures from mat-vis
  - Material.vis.textures lazy fetch
  - to_threejs with live texture data URIs
  - TOML material with vis mapping (stainless finishes)
  - discover (via vis.search, tier-free)
  - Multi-material fetch (light prefetch test)
  - resolve() with texture and scalar fallback

test_catalog.py (8 tests):
  - Root index, category dirs, material pages
  - Vis section, composition, uncertainty rendering
  - Category index table format
  - Total page count (>90)

178 passed (excl e2e), 25 skipped.
- MatVisClient, seed_indexes now re-exported
- vis.adapters module exposed — new adapters (to_ktx2, etc.)
  available automatically when mat-vis-client updates
- vis.MatVisClient().tiers() / .sources() for discovery
- No code changes needed in mat when mat-vis adds new tiers,
  sources, or adapters

178 passed, 25 skipped.
vis.client() returns the shared MatVisClient singleton. Any new
methods mat-vis-client adds (tiers, sources, search, fetch,
new formats) are immediately accessible without pymat changes:

    c = vis.client()
    c.tiers()        # ['128', '1k', '256', '512', '2k']
    c.sources()      # ['ambientcg', 'gpuopen', 'polyhaven']
    c.search("metal")
    c.fetch_all_textures("ambientcg", "Metal032", tier="ktx2")  # when available

178 passed, 25 skipped.
Tests verify the full pipeline:
  pymat.Material → vis.textures → to_threejs → ocp_vscode → screenshot

Skipped by default (MAT_VIS_SKIP_VISUAL=1). Requires:
  build123d, ocp_vscode, playwright + chromium

Three test levels:
1. Material adapter output (JSON dict validation)
2. Material with mat-vis textures (data URI verification)
3. Headless screenshot via ocp_vscode standalone + Playwright
   (placeholder — full wiring TBD)

Visual baselines stored in tests/visual_baselines/.
TestAdapterOutput (2 tests, run with MAT_VIS_SKIP_VISUAL=0):
  - Steel: to_threejs → metalness=1.0, roughness=0.3 (scalars from .vis)
  - Textured metal: to_threejs → 5 base64 PNG data URIs (color,
    normal, roughness, metalness, ao) from live mat-vis v2026.04.0

TestFullPipeline (3 tests, gated by MAT_VIS_SKIP_HEADLESS):
  - Steel cube, wood plank, glass sphere via ocp_vscode standalone
  - Requires ocp_vscode with built JS assets (not available in
    editable dev install — needs PyPI install or yarn build)
  - Framework working, screenshots blank until JS bundle served

The adapter JSON output (visual_output/metal_textured_threejs.json)
proves the full data flow:
  pymat.Material → .vis → mat-vis HTTP range read → to_threejs
  → MeshPhysicalMaterial dict with texture data URIs

This is exactly what three-cad-viewer's material-factory.ts consumes.
5 headless render tests via Playwright + Three.js + SwiftShader:
  - Steel cube (metallic=1.0, roughness=0.3)
  - Red sphere (dielectric, roughness=0.5)
  - Gold cylinder (metallic=1.0, roughness=0.2)
  - Glass sphere (transmission=0.9, ior=1.52)
  - Multi-material assembly (wood base + chrome pin)

Full pipeline validated:
  pymat.Material → .vis scalars → build123d export_gltf
  → Three.js GLTFLoader (headless Chrome + SwiftShader)
  → WebGL MeshPhysicalMaterial render → canvas.toDataURL → PNG

Key fix: build123d exports in meters, Three.js viewer scales
×1000 for visibility.

Tests skip by default (MAT_VIS_SKIP_VISUAL=1). Run with:
  MAT_VIS_SKIP_VISUAL=0 pytest tests/test_visual_regression.py -v

Also includes 2 adapter-output tests (no rendering needed).
pymat/vis/__init__.py:
  Added tags= parameter to search() — require all given tags present
  in entry. Uses ambientcg/polyhaven semantic tags ("brushed", "silver",
  "oak", "concrete") for much better matches than category alone.

metals.toml:
  Curated [vis] mappings for 5 metals based on tag matching:
    stainless: brushed=Metal012, polished=Metal049A, dirty=Metal049B
    aluminum:  smooth=Metal049A, machined=Metal055A
    copper:    polished=Metal043A, oxidized=Metal043B, aged=Metal026
    titanium:  smooth=Metal049A
    brass:     polished=Metal008, oxidized=Metal035

scripts/enrich_vis.py:
  Rewrote to use tag-based matching. Produces more accurate proposals.

scripts/generate_catalog.py:
  Fetches 128px thumbnails from mat-vis when vis.source_id is set.
  4 thumbnails committed for aluminum, brass, copper, titanium.

docs/catalog/:
  Regenerated with the new mappings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant