2025 Second Update

[mdekstrand]

Second major feature update + bugfix release for the 2025 series! Many changes, see the milestone and release notes for more.

Merged PRs

• Simplify test workfow by @mdekstrand in Simplify test workfow #666
• Add StochasticTopNSampler to be correct version of "softmax" sampler by @mdekstrand in Add StochasticTopNSampler to be correct version of "softmax" sampler #667
• Add flexible Torch matrix factorizer by @mdekstrand in Add flexible Torch matrix factorizer #668
• Export configuration classes from lenskit.basic by @mdekstrand in Export configuration classes from lenskit.basic #672
• Improve logging in Ray cluster setups by @mdekstrand in Improve logging in Ray cluster setups #673
• Limit test users to only have users who had item in the train data. by @sushobhan2024 in Limit test users to only have users who had item in the train data. #676
• Add train and recommend commands to the CLI by @mdekstrand in Add train and recommend commands to the CLI #677
• Improve NumPy -> Torch conversions for read-only data by @mdekstrand in Improve NumPy -> Torch conversions for read-only data #680
• Reduce excess matrix manipulations in item-KNN by @mdekstrand in Reduce excess matrix manipulations in item-KNN #678
• Improve progress bar update API by @mdekstrand in Improve progress bar update API #681
• Update FlexMF parameters to better defaults by @mdekstrand in Update FlexMF parameters to better defaults #682
• Run Doctor under coverage by @mdekstrand in Run Doctor under coverage #683
• Use uv for development environments instead of pixi by @mdekstrand in Use uv for development environments instead of pixi #700
• Add lenskit.state module and move the ParameterContainer interface by @mdekstrand in Add lenskit.state module and move the ParameterContainer interface #702
• Create and use separate ModelTrainer objects by @mdekstrand in Create and use separate ModelTrainer objects #701
• Support training checkpointing in ALS by @mdekstrand in Support training checkpointing in ALS #704
• Add CPU & GPU groups by @mdekstrand in Add CPU & GPU groups #706
• Improve logging and reduce sorting in dataset relationships by @mdekstrand in Improve logging and reduce sorting in dataset relationships #707
• Improve debug logging and add xopen to CLI by @mdekstrand in Improve debug logging and add xopen to CLI #708
• Introduce WARP loss to FlexMF by @mdekstrand in Introduce WARP loss to FlexMF #709
• Build with setuptools instead of hatch by @mdekstrand in Build with setuptools instead of hatch #710
• Add better compression support to CLI by @mdekstrand in Add better compression support to CLI #712
• Update pre-commit hook versions by @mdekstrand in Update pre-commit hook versions #713
• Add Rust extension infrastructure and accelerate ItemKNN by @mdekstrand in Add Rust extension infrastructure and accelerate ItemKNN #715
• Replace just with invoke for development tasks by @mdekstrand in Replace just with invoke for development tasks #716
• Add versions to dataset schemas by @mdekstrand in Add versions to dataset schemas #721
• Add SparseRow extension type for CSRs in Arrow by @mdekstrand in Add SparseRow extension type for CSRs in Arrow #722
• Use Rust to accelerate negative sampling by @mdekstrand in Use Rust to accelerate negative sampling #723
• Re-add progress bars and signal checking to Item KNN builder by @mdekstrand in Re-add progress bars and signal checking to Item KNN builder #724
• Add ndarray dep and use it for sampling by @mdekstrand in Add ndarray dep and use it for sampling #725
• Use Maturin to simplify build setup by @mdekstrand in Use Maturin to simplify build setup #726
• Reduce base structure memory consumption by @mdekstrand in Reduce base structure memory consumption #728
• Add Rust acceleration to User KNN by @mdekstrand in Add Rust acceleration to User KNN #730
• Rewrite ALS solvers in Rust by @mdekstrand in Rewrite ALS solvers in Rust #731
• Rewrite FunkSVD in Rust by @mdekstrand in Rewrite FunkSVD in Rust #732
• Build with ABI3 and maturin-action by @mdekstrand in Build with ABI3 and maturin-action #733
• Add SPEC-1 (Lazy Loading) and incorporate version numbers into docs by @mdekstrand in Add SPEC-1 (Lazy Loading) and incorporate version numbers into docs #736
• Implement is_sorted for matrix tables in Rust by @mdekstrand in Implement is_sorted for matrix tables in Rust #737
• Correctly detect no-genre movies in MovieLens by @mdekstrand in Correctly detect no-genre movies in MovieLens #738
• Adding Mean Average Precision by @albus-droid in Adding Mean Average Precision #739
• Add tests to avoid overflow in softmax stochastic sampling by @mdekstrand in Add tests to avoid overflow in softmax stochastic sampling #742
• Add configurable weighting models by @mdekstrand in Add configurable weighting models #744
• Parallelize item-item scoring by @mdekstrand in Parallelize item-item scoring #745
• Support comma-separated lists for uniform nested parallelism configuration by @mdekstrand in Support comma-separated lists for uniform nested parallelism configuration #746
• Test and fix crash on Linux ARM by @mdekstrand in Test and fix crash on Linux ARM #748
• Update ALS solve to zero rows for no-info users/items by @mdekstrand in Update ALS solve to zero rows for no-info users/items #749
• Add importers for UCSD Amazon data by @mdekstrand in Add importers for UCSD Amazon data #751
• refactor FlexMF with less string compares by @mdekstrand in refactor FlexMF with less string compares #752
• Add sklearn NMF as a scorer by @FroggoLight in Add sklearn NMF as a scorer #741
• Speed up ItemList isin checks and evaluation metrics by @mdekstrand in Speed up ItemList isin checks and evaluation metrics #753
• Speed up ItemList indexing by @mdekstrand in Speed up ItemList indexing #754
• Add Gini coefficient metrics by @mdekstrand in Add Gini coefficient metrics #755
• Speed up weight computations by @mdekstrand in Speed up weight computations #756
• Speed up weighted metrics by @mdekstrand in Speed up weighted metrics #757
• Add ItemList.top_n and speed up ItemList operations by @mdekstrand in Add ItemList.top_n and speed up ItemList operations #758
• Add subprocess progress reporting by @mdekstrand in Add subprocess progress reporting #759
• Add a rate limiter and use ray.wait to manage Ray concurrency by @mdekstrand in Add a rate limiter and use ray.wait to manage Ray concurrency #763
• Switch back to Just + docopt by @mdekstrand in Switch back to Just + docopt #764
• Add lenskit data subset command, improve ItemListCollection conversion, and support random seeds in the CLI by @mdekstrand in Add lenskit data subset command, improve ItemListCollection conversion, and support random seeds in the CLI #773
• Force Python UTF-8 mode for license hader update in pre-commit by @mdekstrand in Force Python UTF-8 mode for license hader update in pre-commit #774
• Refactored batch accuracy tests into standard suite by @FroggoLight in Refactored batch accuracy tests into standard suite #776
• Use pytest-retry to retry the flaky stochastic ranker test by @mdekstrand in Use pytest-retry to retry the flaky stochastic ranker test #783
• Add component input hooks to the pipeline by @mdekstrand in Add component input hooks to the pipeline #782
• Add additional tests to verify training item filtering by @mdekstrand in Add additional tests to verify training item filtering #788
• Use 64-bit row pointers in matrix relationship sets by @mdekstrand in Use 64-bit row pointers in matrix relationship sets #789
• Improve dataset documentation by @mdekstrand in Improve dataset documentation #790
• Increaes minimum PyArrow vesion by @mdekstrand in Increaes minimum PyArrow vesion #791
• Add LensKit settings and power querying by @mdekstrand in Add LensKit settings and power querying #794
• Revise documentation to be main site by @mdekstrand in Revise documentation to be main site #798
• Add devcontainer and update development dependencies by @mdekstrand in Add devcontainer and update development dependencies #799
• Automatically search up for lenskit.toml in CLI by @mdekstrand in Automatically search up for lenskit.toml in CLI #802
• Fix syntax in testsetup for item-list docs by @mdekstrand in Fix syntax in testsetup for item-list docs #805
• added binarize_ratings function in DatasetBuilder by @vibhor-5 in added binarize_ratings function in DatasetBuilder #803
• Fix locate_configuration_root and Conda tests by @mdekstrand in Fix locate_configuration_root and Conda tests #808
• Drop PyTorch max and update locked dependencies by @mdekstrand in Drop PyTorch max and update locked dependencies #807
• Work around Ray extension type serialization errors by @mdekstrand in Work around Ray extension type serialization errors #809
• Use Rust-based hash table instead of Pandas index to implement Vocabulary by @mdekstrand in Use Rust-based hash table instead of Pandas index to implement Vocabulary #811
• Refactor indirect hashtables to support reuse by @mdekstrand in Refactor indirect hashtables to support reuse #813
• Use uvx --isolated for compat tests by @mdekstrand in Use uvx --isolated for compat tests #816
• Use mise tasks instead of just by @mdekstrand in Use mise tasks instead of just #814
• Implement a CoordinateTable structure for fast duplicate checking by @mdekstrand in Implement a CoordinateTable structure for fast duplicate checking #815
• Improve recommend CLI for batch inference, output, profiling by @mdekstrand in Improve recommend CLI for batch inference, output, profiling #817
• Fix coordinate table breakage when adding interactions after filtering or removing by @mdekstrand in Fix coordinate table breakage when adding interactions after filtering or removing #820
• Simplify PyPI CUDA setup by @mdekstrand in Simplify PyPI CUDA setup #824
• Implement usable pipeline configuration files and CLI support by @mdekstrand in Implement usable pipeline configuration files and CLI support #831
• Implement faster negative sampling and use CoordinateTable by @mdekstrand in Implement faster negative sampling and use CoordinateTable #832
• More FlexMF performance improvements by @mdekstrand in More FlexMF performance improvements #840
• Add parallel config to lenskit doctor by @mdekstrand in Add parallel config to lenskit doctor #842
• Add support for repeated interactions by @FroggoLight in Add support for repeated interactions  #812
• Make list metric DF have a standard Index when there is one key field by @mdekstrand in Make list metric DF have a standard Index when there is one key field #845
• Add torch-geometric and LightGCN by @mdekstrand in Add torch-geometric and LightGCN #822
• Unify list and decomposed metric interfaces by @sushobhan2024 in Unify list and decomposed metric interfaces #792
• Fix: correct attribute-to-entity mapping by @samiravaez in Fix: correct attribute-to-entity mapping #850
• Fix Conda builds + GNN tests by @mdekstrand in Fix Conda builds + GNN tests #852
• Fix configuration warning on pipelines with placeholders by @mdekstrand in Fix configuration warning on pipelines with placeholders #853
• Update and fix pre-commit hooks by @mdekstrand in Update and fix pre-commit hooks #855
• Refactor embedding size configuration for consistency + exponential support by @mdekstrand in Refactor embedding size configuration for consistency + exponential support #851
• Add validation for entity class name and attribute name by @FroggoLight in Add validation for entity class name and attribute name  #854
• Defer component instantiation when loading configurations by @mdekstrand in Defer component instantiation when loading configurations #858
• Add basic data-splitting CLI by @mdekstrand in Add basic data-splitting CLI #859
• Add experimental hyperparameter tuning library and CLI by @mdekstrand in Add experimental hyperparameter tuning library and CLI #857
• Enable CI tests for the hyperparameter tuning by @mdekstrand in Enable CI tests for the hyperparameter tuning #861
• ALS: fast-path very large scoring by @mdekstrand in ALS: fast-path very large scoring #862
• Add --save-pipeline option to tune CLI by @mdekstrand in Add --save-pipeline option to tune CLI #863
• Add Fair Re-Ranker (FA*IR) by @samiravaez in Add Fair Re-Ranker (FA*IR) #860
• Add tuning settings to LensKit config + support a GPU resource multiplier by @mdekstrand in Add tuning settings to LensKit config + support a GPU resource multiplier #864
• Make tuning submodules private by @mdekstrand in Make tuning submodules private #868
• Add tune extra + update release notes by @mdekstrand in Add tune extra + update release notes #870

New Contributors

• @sushobhan2024 made their first contribution in Limit test users to only have users who had item in the train data. #676
• @albus-droid made their first contribution in Adding Mean Average Precision #739
• @FroggoLight made their first contribution in Add sklearn NMF as a scorer #741
• @vibhor-5 made their first contribution in added binarize_ratings function in DatasetBuilder #803

Full Changelog: v2025.2.0...v2025.3.0

---

This discussion was created from the release 2025 Second Update.