Skip to content

Releases: opensanctions/yente

v5.3

01 Apr 09:34
@leonhandreke leonhandreke
v5.3.0
f10454b

Choose a tag to compare

View all tags
v5.3 Latest
Latest

Note: Triggers full index rebuild

This release contains the usual round of upgrades to our whole stack, as well as the following changes and improvements:

  • we've removed the matcher and features fields from /match API responses. Information equivalent to the matcher field is available via the /algorithms route. To replace features, the explanations field has been added a while ago that is a strict superset of features. In addition to the partial scores produced by each component of the matching system, it includes textual descriptions of the matching decisions from each of these subsystems.
  • a bunch of improvements around failures in clone_index, which is used when yente updates its index to a new version using delta files. A failed clone would previously leave the system in a stuck state which would have to be fixed manually. Thanks @dimoschi for finding this bug, investigating and collaborating on the resolution!

Just as a heads-up: Elasticsearch 8 is nearing its end of life in January 2027. We will release the last version of yente that is compatible with Elasticsearch 8 servers in May 2026. We've added a section on that transition in the documentation. TL;DR: upgrade your Elastic server to 8.19.13 first, restart Elastic, then upgrade to 9.x, restart Elastic, and all should go smoothly - please let us know if not! Consequently, we've set the version of the Elastic server in the docker-compose.yml to 8.19.13 for now.

As usual, this release contains updates to our whole stack, including an upgrade of the cryptography package to 46.0.6.

What's Changed

New Contributors

Full Changelog: v5.2.0...v5.3.0

Contributors

pudo, leonhandreke, and 3 other contributors
Assets 2
Loading

v5.2.0

28 Jan 10:43
@leonhandreke leonhandreke
v5.2.0
98258e1

Choose a tag to compare

View all tags
v5.2.0

Note: Triggers full index rebuild

This release brings a bunch of improvements that we've made in the aftermath of releasing the logic-v2 scoring algorithm. Some little tweaks to how things are scored as well as new configuration options (for example, one to exclude aliases from name matching). For names, in addition to the detailed explanation on how the name was matched, we now include the names of the query and the result entity that were used to make the match. We've also made some performance optimizations to logic-v2 and have made it so that requests can be scored concurrently in yente, keeping response times more even in high-load scenarios.

We've spent some time to make sure that yente works with Amazon OpenSearch Serverless (AOSS), an AWS product that provides a fully managed search index that serves as a backend for yente. Please refer to the documentation for how to configure AOSS as the search provider. That said, we don't have any real-life experience running this configuration in production -— if you have problem reports or success stories running yente with OpenSearch Serverless, please let us know!

This release contains two breaking changes for everyone running yente using the example docker-compose.yml that is shipped with yente:

  • First, yente is now configured to use delivery.opensanctions.org as a data retrieval location by default, encouraging prospective commercial users to create a delivery token — see our documentation for more information. Setting this up only takes a minute and nothing about our licensing terms is changing. The idea here is not to create a wall around our data, but to steer our commercial customers towards a mechanism that allows us to better support them by collecting telemetry, and to provide preferred service to them in case things go south in a technical or legal way for the public data bucket. Non-commercial users and users exempted from commercial licensing can easily configure yente to access the public data archive directly.
  • Further, the app service now depends on the index, meaning that docker compose up -d app will now also bring up an index by default (see c0f5b71).

As usual, this release contains updates to our whole stack, including an upgrade of the cryptography package from 46.0.2 to 46.0.4.

What's Changed

  • build(deps): bump aiohttp[speedups] from 3.12.15 to 3.13.0 by @dependabot[bot] in #908
  • match: log dataset, threshold, algorithm as passed in request by @leonhandreke in #911
  • build(deps): bump fastapi from 0.118.0 to 0.119.0 by @dependabot[bot] in #913
  • build(deps): bump aiofiles from 24.1.0 to 25.1.0 by @dependabot[bot] in #910
  • build(deps): bump structlog from 25.4.0 to 25.5.0 by @dependabot[bot] in #931
  • build(deps): bump uvicorn[standard] from 0.37.0 to 0.38.0 by @dependabot[bot] in #919
  • build(deps): bump orjson from 3.11.3 to 3.11.4 by @dependabot[bot] in #928
  • Allow concurrency between scoring calls by @pudo in #929
  • build(deps): bump fastapi from 0.119.0 to 0.120.1 by @dependabot[bot] in #930
  • build(deps): bump aiohttp[speedups] from 3.13.0 to 3.13.1 by @dependabot[bot] in #920
  • build(deps): bump cryptography from 46.0.2 to 46.0.3 by @dependabot[bot] in #918
  • build(deps): update types-aiofiles requirement from <25.0,>=24.0 to >=24.0,<26.0 by @dependabot[bot] in #914
  • deps: Get rid of jellyfish, we don't depend on it (anymore?) by @leonhandreke in #915
  • build(deps): bump elasticsearch[async] from 8.19.1 to 8.19.2 by @dependabot[bot] in #933
  • build(deps): bump aiohttp[speedups] from 3.13.1 to 3.13.2 by @dependabot[bot] in #932
  • build(deps): bump fastapi from 0.120.1 to 0.120.3 by @dependabot[bot] in #938
  • Nomenklatura with nm_name_property & new FtM by @pudo in #934
  • build(deps): bump fastapi from 0.120.3 to 0.120.4 by @dependabot[bot] in #939
  • build(deps): bump fastapi from 0.120.4 to 0.121.0 by @dependabot[bot] in #941
  • Wait for index before starting app in docker compose by @jbothma in #945
  • build(deps): bump fastapi from 0.121.0 to 0.121.1 by @dependabot[bot] in #946
  • build(deps): bump pyicu from 2.15.3 to 2.16 by @dependabot[bot] in #944
  • build(deps): bump rigour from 1.4.0 to 1.4.2 by @dependabot[bot] in #948
  • build(deps): bump fastapi from 0.121.1 to 0.121.2 by @dependabot[bot] in #951
  • build(deps): bump nomenklatura from 4.1.10 to 4.2.0 by @dependabot[bot] in #949
  • build(deps): bump fastapi from 0.121.2 to 0.121.3 by @dependabot[bot] in #954
  • opensearch: some fixes to make Amazon OpenSearch Serverless work by @leonhandreke in #937
  • build(deps): bump opensearch-py[async] from 3.0.0 to 3.1.0 by @dependabot[bot] in #956
  • match: Switch default and best algorithms to logic-v2 by @leonhandreke in #961
  • build(deps): bump actions/checkout from 5 to 6 by @dependabot[bot] in #955
  • build(deps-dev): bump pytest from 8.4.2 to 9.0.1 by @dependabot[bot] in #950
  • Raise an exception if scope not found in defined catalog by @leonhandreke in #923
  • doc(index.md):fix double slash in hosted API link by @mainibet in #995
  • docs(settings.md): update YENTE_QUERY_CONCURRENCY default to 50 by @mainibet in #996
  • build(deps): bump uvicorn[standard] from 0.38.0 to 0.40.0 by @dependabot[bot] in #981
  • build(deps): bump anyio from 4.11.0 to 4.12.1 by @dependabot[bot] in #994
  • build(deps): bump elasticsearch[async] from 8.19.2 to 8.19.3 by @dependabot[bot] in #983
  • build(deps-dev): bump pytest from 9.0.1 to 9.0.2 by @dependabot[bot] in #971
  • build(deps): bump aiohttp[speedups] from 3.13.2 to 3.13.3 by @dependabot[bot] in #990
  • build(deps): bump python-multipart from 0.0.20 to 0.0.21 by @dependabot[bot] in #977
  • FtM stack upgrade by @leonhandreke in #1000
  • Modernize pyproject.toml a bit and advise use of uv by @leonhandreke in #1002
  • build(deps): bump orjson from 3.11.4 to 3.11.5 by @dependabot[bot] in #970
  • build(deps): bump fastapi from 0.121.3 to 0.128.0 by @dependabot[bot] in #986
  • Configure yente with delivery.opensanctions.com as default by @leonhandreke in #924
  • build(deps): bump python-multipart from 0.0.21 to 0.0.22 by @dependabot[bot] in #1003
  • build(deps): bump cryptography from 46.0.3 to 46.0.4 by @dependabot[bot] in #1006
  • build(deps): bump rigour from 1.6.1 to 1.6.2 by @dependabot[bot] in #1007
  • docs: Add monitoring docs for /healthz, /readyz, /catalog by @leonhandreke in #1005

Full Changelog: v5.0.2...v5.2.0

Contributors

pudo, leonhandreke, and 3 other contributors
Loading

v5.0.2

06 Oct 12:41
@pudo pudo
v5.0.2
96a0e61

Choose a tag to compare

View all tags
v5.0.2

Brings in updates to symbolic matching in nomenklatura and rigour, and tries to make single-word queries less horrible by forcing fuzzy matching on them. Clarify and raise for invalid catalog specifications.

What's Changed

  • build(deps): update click requirement from ~=8.2.0 to >=8.2,<8.4 by @dependabot[bot] in #895
  • build(deps): bump fastapi from 0.116.2 to 0.117.1 by @dependabot[bot] in #897
  • build(deps): bump anyio from 4.10.0 to 4.11.0 by @dependabot[bot] in #899
  • build(deps): bump aiocsv from 1.3.2 to 1.4.0 by @dependabot[bot] in #898
  • build(deps): bump uvicorn[standard] from 0.35.0 to 0.37.0 by @dependabot[bot] in #900
  • build(deps): bump fastapi from 0.117.1 to 0.118.0 by @dependabot[bot] in #901
  • build(deps): bump cryptography from 46.0.1 to 46.0.2 by @dependabot[bot] in #903
  • Log HTTP error URL explicitly by @jbothma in #906
  • Enable fuzzy search for single-word queries by @pudo in #907
  • Clarify multi catalog/dataset manifest configuration by @jbothma in #904

Full Changelog: v5.0.1...v5.0.2

Contributors

pudo, jbothma, and dependabot
Loading

v5.0.1

23 Sep 14:29
@leonhandreke leonhandreke
v5.0.1
3250024

Choose a tag to compare

View all tags
v5.0.1

Just a minor patch release of yente, pulling in a new nomenklatura with some fixes to the new logic-v2 scoring algorithm. Thank you to @baldurh for the in-depth feedback and bug hunting! More feedback on this new scoring system (and all other aspects of yente is always appreciated - please get in touch here on GitHub or our Forum.

Note that this release also bumps cryptography from 45.0.7 to 46.0.1.

Full Changelog: v5.0.0...v5.0.1

Contributors

baldurh
Loading

v5.0.0

15 Sep 15:00
@leonhandreke leonhandreke
v5.0.0
5fc6b9a

Choose a tag to compare

View all tags
v5.0.0
  • New logic-v2 matcher: We're including a new matching system, logic-v2. This new system reflects feedback from users of logic-v1 and introduces a more precise, explainable, and culturally-aware way to match names of people and companies. logic-v2 reduces false positives, is fully deterministic, improves cross-language and cross-script matching, runs fast, and provides detailed explanations of its decision-making path. Please be aware that we'll still be adjusting logic-v2 based on user feedback, so don't expect every scoring decision to be set in stone yet. We look forward to your feedback, be it as an issue report on our GitHub, or on our discussion forum or via the support team.
  • Field deprecations and renames:
    • The matcher field in /match responses will be removed in a future version of yente. Equivalent information is available via the /algorithms endpoint.
    • The /match endpoint response has gained a new explanations field that is a strict superset of features. In addition to the partial scores produced by each component of the matching system, it includes textual descriptions of the matching decisions from each of these subsystems. These descriptions are designed for display in analyst workbenches or can be passed to generative AI tools to help interpret screening alerts. The features field will be removed in a future version of yente.
    • The cutoff parameter on the /match endpoint has been deprecated and will be removed in a future version of yente. If you care about low-scoring results being returned, please set the threshold parameter instead.
    • The /algorithms endpoint has gained a new docs field that is a strict superset of features. In addition to a documentation of the algorithm features (yes, those featured in the explanations field in /match), it contains a list of configuration knobs to tune the algorithm to your individual needs. Currently, only logic-v2 offers these knobs.
    • These algorithm configuration options are passed in a new config field on the /match request.
  • Locking mechanism to prevent concurrent reindexes: One of the most common issues with yente, both in our hosted deployment an for external users, was two re-index jobs stepping on each other's toes or overloading the Elastic backend, doing the same thing twice. Before we start a reindex, we now write a little lock to the Elastic backend so that other instances of yente who might be planning to do a reindex know that one is already in progress. Regardless, if you're running multiple instances of yente, you should still configure a separate re-indexing cronjob. See documentation on how to deploy yente for more information.
  • Audit log of index operations: A log of re-indexing and index cleanup operations is written to a special index in Elastic. This allows users to get an accurate record of what data became available when in their yente without digging through log messages. To read the audit log, simply run yente audit-log (or use another CSV viewer of your choice).
  • Google Cloud Logging compatible request logging: If you're running yente on Google Cloud Run, handy little badges with information about the HTTP request as well as a little button to filter logs from only this trace will show up in the Logs Explorer. Careful: the format of the log message logged for each request changed, so if you're parsing the JSON logged by yente, please update your infrastructure accordingly.
  • FollowTheMoney 4.0: This release pulls in an updated version of the data model underpinning yente. If you're ingesting custom data sources that you're generating yourself, please check out the release notes. The only change that may be notable for users of yente is probably the rename of CryptoWallet:managingExchange. Users relying on recent published OpenSanctions data can ignore this change.
  • Documentation moved to yente.followthemoney.tech: yente is part of the wider open source ecosystem around the FollowTheMoney data model, and the move of its documentation reflects this. The documentation is now part of the source repository and can be edited by anyone. Documentation is never perfect, and we welcome your PRs! The OpenSanctions website remains a good place to read about the data underpinning yente and our hosted API.
  • Celebrity-friendly scoring in /search: While /match is at the heart of what yente does best, the /search endpoint is what users usually hit first when they type in a search query on opensanctions.org. By deploying cutting-edge research from our search engine labs, we ensure the Putin ranks first when you type in "putin". Much wow!
  • The /updatez endpoint is now disabled by default. Set the UPDATE_TOKEN authentication token to a secret value to enable it.
  • The usual round of dependency upgrades, among them an upgrade of cryptography from 45.0.5 to 45.0.6
Loading

v4.5.1

07 Aug 14:30
@leonhandreke leonhandreke
v4.5.1
c524bc6

Choose a tag to compare

View all tags
v4.5.1

Some minor dependency upgrades, plus a little convenience manifest file that allows OpenSanctions customers to easily supply the secret delivery token for the delivery.opensanctions.com service.

What's Changed

Full Changelog: v4.5.0...v4.5.1

Contributors

pudo, leonhandreke, and dependabot
Loading

v4.5.0

09 Jul 13:46
@leonhandreke leonhandreke
v4.5.0
832160d

Choose a tag to compare

View all tags
v4.5.0

Note: Triggers full index rebuild

I'm happy to announce the release of Yente 4.5.0 with a few new features and the usual bag of dependency upgrades. This release includes the following new features:

  • a new auth_token parameter to set on catalogs in the manifest. This token (which can also be set from an environment variable) will be sent in the Authorization header in all requests for the catalog and its dataset files.
    • This mechanism replaces the DATA_TOKEN environment variable (which sets the Authentication header). This mechanism is now deprecated and will be removed in a future version of Yente. If you're using this, please migrate to auth_token instead.
  • The /match endpoint now has a exclude_entity_ids parameter that allows callers to exclude some entities from matching. This may be useful if you're doing periodic screening of the same entities and have decided that a match is a false-positive and want it to be excluded from matching completely. Thanks to @baldurh for the idea!

Further, the release includes the following improvements & changes worth noting:

  • Entities are now retrieved from the search index with stable ordering during search and match queries. This produces reproducible match results in cases where all match candidates equally ranked in terms of relevance, often because of an overly broad query (like "LLC", "John") matching too many entities. It does not, in itself, improve the quality of candidates that will be retrieved.
  • The usual round of dependency upgrades, including the cryptography package from 45.0.3 to 45.0.5

Full Commit Log

Full Changelog: v4.4.0...v4.5.0

Contributors

leonhandreke, jbothma, and 2 other contributors
Loading

v4.4.0

05 Jun 09:41
@jbothma jbothma
v4.4.0
c2eb034
This commit was signed with the committer’s verified signature.
jbothma JD Bothma
SSH Key Fingerprint: V+leRqZL4qPsUuJDCQRdjM2EVwcYLexoRUqsPdQe2SE
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v4.4.0

Note: Triggers full index rebuild

  • The tokenised name indexing has been improved - prefixes from Person names, e.g. Mrs, Mr, etc are stripped. This should reduce the false positive rate for cases where those prefixes contributed to the score.
  • The environment variable YENTE_MATCH_FUZZY will take effect again. When true (default), the candidate generation stage (Elasticsearch query) of /match queries can include names with small spelling differences from the query. This does not affect the match score as indicated in the response.

What's Changed

Full Changelog: v4.3.1...v4.4.0

Contributors

pudo, jbothma, and dependabot
Loading

v4.3.1

10 Apr 08:14
@jbothma jbothma
v4.3.1
2aa4e44
This commit was signed with the committer’s verified signature.
jbothma JD Bothma
SSH Key Fingerprint: V+leRqZL4qPsUuJDCQRdjM2EVwcYLexoRUqsPdQe2SE
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v4.3.1

Note: Triggers full index rebuild - Due to index schema changes this update triggers a full index rebuild even if you are up to date and use incremental index updates. The changes are backward-compatible so your yente service should remain available during rebuild, but the typical full index rebuild load will be seen on your ElasticSearch/OpenSearch deployment.

What's Changed

This fixes a couple of bugs in the (beta) adjacent entities API

  • Adjacent entities not found if the root is an edge
  • Adjacent entities not found if the property name conflicts with a property of another type (observed with directors' directorships)

#718

Other changes:

Full Changelog: v4.3.0...v4.3.1

Contributors

jbothma and dependabot
Loading

v4.3.0

04 Apr 10:33
@leonhandreke leonhandreke
v4.3.0
2c3cc90

Choose a tag to compare

View all tags
v4.3.0

In addition to the usual routine dependency upgrades and fixes, this release includes two new endpoints to retrieve entities adjacent to a given entity (e.g. holders of an office or assets owned by a person) in a paginated way.

Adjacent entities API

This release includes a beta version of two new adjacent entities API endpoints. These APIs may still be subject to change, a production-ready stable release will be announced in the future. We would love to get your feedback, here on GitHub or on our Discourse or any other way.

The existing /entities/{entity_id}endpoint can nest adjacent entities (using the nested=true query parameter), but does so without limits of how many entities can be included. This can lead to large, slow responses when requesting entities with a large number of adjacent entities, such as institutions with many securities, or PEP positions with many holders. The /entities/{entity_id}/adjacent and /entities/{entity_id}/adjacent/{property_name} endpoints return a limited number of results by default with pagination parameters for retrieving more.

What's Changed

New Contributors

Full Changelog: v4.2.4...v4.3.0

Contributors

leonhandreke, jbothma, and dependabot
Loading