Skip to content

Commit ba6f4b5

Browse files
authored
H-3692: Move deer to labs (#68)
* chore: move deer * chore: remove superfluos files * fix: suggestions from code review * fix: repository links
1 parent 85f104b commit ba6f4b5

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

103 files changed

+14737
-0
lines changed

libs/deer/CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Changelog
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6+
7+
## [Unreleased]

libs/deer/Cargo.toml

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
[package]
2+
name = "deer"
3+
version = "0.0.0-reserved"
4+
authors = { workspace = true }
5+
edition = "2021"
6+
rust-version = "1.65"
7+
license = "MIT OR Apache-2.0"
8+
description = "A backend-agnostic fail-slow deserialization framework"
9+
documentation = "https://docs.rs/deer"
10+
repository = "https://github.com/hashintel/labs/tree/main/libs/deer"
11+
keywords = ["deserialize", "serde", "no_std"]
12+
categories = ["no-std", "rust-patterns"]
13+
exclude = ["package.json"]
14+
publish = false
15+
16+
[dependencies]
17+
# Public workspace dependencies
18+
error-stack = { workspace = true, public = true, default-features = false, features = ["unstable"] }
19+
20+
# Public third-party dependencies
21+
erased-serde = { workspace = true, public = true, features = ['alloc'] }
22+
num-traits = { workspace = true, public = true }
23+
serde = { workspace = true, public = true, features = ['alloc', 'derive'] }
24+
25+
# Private workspace dependencies
26+
27+
# Private third-party dependencies
28+
29+
[dev-dependencies]
30+
approx = { workspace = true }
31+
deer-desert = { path = "./desert", features = ['pretty'] }
32+
paste = { workspace = true }
33+
proptest = { workspace = true, features = ["std"] }
34+
seq-macro = { workspace = true }
35+
serde_json = { workspace = true, features = ['arbitrary_precision'] }
36+
similar-asserts = { workspace = true, features = ["serde"] }
37+
38+
[build-dependencies]
39+
rustc_version = { workspace = true }
40+
41+
[features]
42+
default = ['std']
43+
std = ['serde/std', 'error-stack/std']
44+
arbitrary-precision = []
45+
46+
[lints]
47+
workspace = true
48+
49+
[package.metadata.sync.turborepo]
50+
ignore = true

libs/deer/LICENSE-APACHE.md

Lines changed: 189 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,189 @@
1+
# Apache License
2+
3+
_Version 2.0, January 2004_
4+
_&lt;<http://www.apache.org/licenses/>&gt;_
5+
6+
### Terms and Conditions for use, reproduction, and distribution
7+
8+
#### 1. Definitions
9+
10+
“License” shall mean the terms and conditions for use, reproduction, and
11+
distribution as defined by Sections 1 through 9 of this document.
12+
13+
“Licensor” shall mean the copyright owner or entity authorized by the copyright
14+
owner that is granting the License.
15+
16+
“Legal Entity” shall mean the union of the acting entity and all other entities
17+
that control, are controlled by, or are under common control with that entity.
18+
For the purposes of this definition, “control” means **(i)** the power, direct or
19+
indirect, to cause the direction or management of such entity, whether by
20+
contract or otherwise, or **(ii)** ownership of fifty percent (50%) or more of the
21+
outstanding shares, or **(iii)** beneficial ownership of such entity.
22+
23+
“You” (or “Your”) shall mean an individual or Legal Entity exercising
24+
permissions granted by this License.
25+
26+
“Source” form shall mean the preferred form for making modifications, including
27+
but not limited to software source code, documentation source, and configuration
28+
files.
29+
30+
“Object” form shall mean any form resulting from mechanical transformation or
31+
translation of a Source form, including but not limited to compiled object code,
32+
generated documentation, and conversions to other media types.
33+
34+
“Work” shall mean the work of authorship, whether in Source or Object form, made
35+
available under the License, as indicated by a copyright notice that is included
36+
in or attached to the work (an example is provided in the Appendix below).
37+
38+
“Derivative Works” shall mean any work, whether in Source or Object form, that
39+
is based on (or derived from) the Work and for which the editorial revisions,
40+
annotations, elaborations, or other modifications represent, as a whole, an
41+
original work of authorship. For the purposes of this License, Derivative Works
42+
shall not include works that remain separable from, or merely link (or bind by
43+
name) to the interfaces of, the Work and Derivative Works thereof.
44+
45+
“Contribution” shall mean any work of authorship, including the original version
46+
of the Work and any modifications or additions to that Work or Derivative Works
47+
thereof, that is intentionally submitted to Licensor for inclusion in the Work
48+
by the copyright owner or by an individual or Legal Entity authorized to submit
49+
on behalf of the copyright owner. For the purposes of this definition,
50+
“submitted” means any form of electronic, verbal, or written communication sent
51+
to the Licensor or its representatives, including but not limited to
52+
communication on electronic mailing lists, source code control systems, and
53+
issue tracking systems that are managed by, or on behalf of, the Licensor for
54+
the purpose of discussing and improving the Work, but excluding communication
55+
that is conspicuously marked or otherwise designated in writing by the copyright
56+
owner as “Not a Contribution.”
57+
58+
“Contributor” shall mean Licensor and any individual or Legal Entity on behalf
59+
of whom a Contribution has been received by Licensor and subsequently
60+
incorporated within the Work.
61+
62+
#### 2. Grant of Copyright License
63+
64+
Subject to the terms and conditions of this License, each Contributor hereby
65+
grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
66+
irrevocable copyright license to reproduce, prepare Derivative Works of,
67+
publicly display, publicly perform, sublicense, and distribute the Work and such
68+
Derivative Works in Source or Object form.
69+
70+
#### 3. Grant of Patent License
71+
72+
Subject to the terms and conditions of this License, each Contributor hereby
73+
grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
74+
irrevocable (except as stated in this section) patent license to make, have
75+
made, use, offer to sell, sell, import, and otherwise transfer the Work, where
76+
such license applies only to those patent claims licensable by such Contributor
77+
that are necessarily infringed by their Contribution(s) alone or by combination
78+
of their Contribution(s) with the Work to which such Contribution(s) was
79+
submitted. If You institute patent litigation against any entity (including a
80+
cross-claim or counterclaim in a lawsuit) alleging that the Work or a
81+
Contribution incorporated within the Work constitutes direct or contributory
82+
patent infringement, then any patent licenses granted to You under this License
83+
for that Work shall terminate as of the date such litigation is filed.
84+
85+
#### 4. Redistribution
86+
87+
You may reproduce and distribute copies of the Work or Derivative Works thereof
88+
in any medium, with or without modifications, and in Source or Object form,
89+
provided that You meet the following conditions:
90+
91+
- **(a)** You must give any other recipients of the Work or Derivative Works a copy of
92+
this License; and
93+
- **(b)** You must cause any modified files to carry prominent notices stating that You
94+
changed the files; and
95+
- **(c)** You must retain, in the Source form of any Derivative Works that You distribute,
96+
all copyright, patent, trademark, and attribution notices from the Source form
97+
of the Work, excluding those notices that do not pertain to any part of the
98+
Derivative Works; and
99+
- **(d)** If the Work includes a “NOTICE” text file as part of its distribution, then any
100+
Derivative Works that You distribute must include a readable copy of the
101+
attribution notices contained within such NOTICE file, excluding those notices
102+
that do not pertain to any part of the Derivative Works, in at least one of the
103+
following places: within a NOTICE text file distributed as part of the
104+
Derivative Works; within the Source form or documentation, if provided along
105+
with the Derivative Works; or, within a display generated by the Derivative
106+
Works, if and wherever such third-party notices normally appear. The contents of
107+
the NOTICE file are for informational purposes only and do not modify the
108+
License. You may add Your own attribution notices within Derivative Works that
109+
You distribute, alongside or as an addendum to the NOTICE text from the Work,
110+
provided that such additional attribution notices cannot be construed as
111+
modifying the License.
112+
113+
You may add Your own copyright statement to Your modifications and may provide
114+
additional or different license terms and conditions for use, reproduction, or
115+
distribution of Your modifications, or for any such Derivative Works as a whole,
116+
provided Your use, reproduction, and distribution of the Work otherwise complies
117+
with the conditions stated in this License.
118+
119+
#### 5. Submission of Contributions
120+
121+
Unless You explicitly state otherwise, any Contribution intentionally submitted
122+
for inclusion in the Work by You to the Licensor shall be under the terms and
123+
conditions of this License, without any additional terms or conditions.
124+
Notwithstanding the above, nothing herein shall supersede or modify the terms of
125+
any separate license agreement you may have executed with Licensor regarding
126+
such Contributions.
127+
128+
#### 6. Trademarks
129+
130+
This License does not grant permission to use the trade names, trademarks,
131+
service marks, or product names of the Licensor, except as required for
132+
reasonable and customary use in describing the origin of the Work and
133+
reproducing the content of the NOTICE file.
134+
135+
#### 7. Disclaimer of Warranty
136+
137+
Unless required by applicable law or agreed to in writing, Licensor provides the
138+
Work (and each Contributor provides its Contributions) on an “AS IS” BASIS,
139+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied,
140+
including, without limitation, any warranties or conditions of TITLE,
141+
NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are
142+
solely responsible for determining the appropriateness of using or
143+
redistributing the Work and assume any risks associated with Your exercise of
144+
permissions under this License.
145+
146+
#### 8. Limitation of Liability
147+
148+
In no event and under no legal theory, whether in tort (including negligence),
149+
contract, or otherwise, unless required by applicable law (such as deliberate
150+
and grossly negligent acts) or agreed to in writing, shall any Contributor be
151+
liable to You for damages, including any direct, indirect, special, incidental,
152+
or consequential damages of any character arising as a result of this License or
153+
out of the use or inability to use the Work (including but not limited to
154+
damages for loss of goodwill, work stoppage, computer failure or malfunction, or
155+
any and all other commercial damages or losses), even if such Contributor has
156+
been advised of the possibility of such damages.
157+
158+
#### 9. Accepting Warranty or Additional Liability
159+
160+
While redistributing the Work or Derivative Works thereof, You may choose to
161+
offer, and charge a fee for, acceptance of support, warranty, indemnity, or
162+
other liability obligations and/or rights consistent with this License. However,
163+
in accepting such obligations, You may act only on Your own behalf and on Your
164+
sole responsibility, not on behalf of any other Contributor, and only if You
165+
agree to indemnify, defend, and hold each Contributor harmless for any liability
166+
incurred by, or claims asserted against, such Contributor by reason of your
167+
accepting any such warranty or additional liability.
168+
169+
_END OF TERMS AND CONDITIONS_
170+
171+
### APPENDIX: Apply the Apache License to a specific file
172+
173+
To apply the Apache License to an individual file, attach the following notice.
174+
The text should be enclosed in the appropriate comment syntax for the file
175+
format.
176+
177+
Copyright © 2022–, HASH
178+
179+
Licensed under the Apache License, Version 2.0 (the "License");
180+
you may not use this file except in compliance with the License.
181+
You may obtain a copy of the License at
182+
183+
http://www.apache.org/licenses/LICENSE-2.0
184+
185+
Unless required by applicable law or agreed to in writing, software
186+
distributed under the License is distributed on an "AS IS" BASIS,
187+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
188+
See the License for the specific language governing permissions and
189+
limitations under the License.

libs/deer/LICENSE-MIT.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# MIT License
2+
3+
Copyright © 2022–, HASH
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

libs/deer/LICENSE.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# License
2+
3+
Licensed under either of the [Apache License, Version 2.0](LICENSE-APACHE.md) or [MIT license](LICENSE-MIT.md) at your option.
4+
5+
For more information about contributing to this crate, see our top-level [CONTRIBUTING](https://github.com/hashintel/labs/blob/main/.github/CONTRIBUTING.md) policy.

libs/deer/README.md

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
[license]: https://github.com/hashintel/labs/blob/main/libs/deer/LICENSE.md
2+
[Apache License, Version 2.0]: https://github.com/hashintel/labs/blob/main/libs/deer/LICENSE-APACHE.md
3+
[MIT License]: https://github.com/hashintel/labs/blob/main/libs/deer/LICENSE-MIT.md
4+
5+
# deer
6+
7+
`deer` is an **experimental** backend-agnostic deserialization framework for Rust, featuring meaningful error messages and context (utilizing [`error-stack`](https://crates.io/crates/error-stack)) and a fail-slow behavior by default.
8+
9+
## Fail-Slow
10+
11+
Currently available Rust deserializers have mostly been developed with correctness and speed in mind. These are universally beneficial optimizations, but in certain cases (such as when collecting user-facing validation feedback) there are relatively few options available within Rust that allow for extended evaluation beyond a single error. `deer` aims to improve this situation by consciously trading off an acceptable degree of speed to enable the surfacing of multiple errors.
12+
13+
## Example
14+
15+
End-user facing APIs are a well-suited example for `deer`.
16+
17+
Given the following example:
18+
19+
```rust
20+
#[derive(Debug, serde::Deserialize, deer::Deserialize)]
21+
struct Body {
22+
u8: u8,
23+
string: String
24+
}
25+
26+
fn main() {
27+
let payload = json!({
28+
"u8": 256,
29+
"string": null,
30+
"extra": 1
31+
});
32+
33+
// Note: Syntax is not final!
34+
let result = deer::json::from_value::<Body>(payload);
35+
let error = result.expect_err("should fail");
36+
println!("{error:?}");
37+
38+
let result = serde_json::from_value::<Body>(payload);
39+
let error = result.expect_err("should fail");
40+
println!("{error:?}");
41+
}
42+
```
43+
44+
`serde` will fail immediately upon encountering that `256` is larger than what `u8` allows. This leads to frustration for the API consumer, as once they fix that issue the next problem, that `string` cannot be null, will be returned. `serde` also does not include path information about where the issue is located, `deer` does!
45+
46+
`deer` solves this problem, by returning every issue present. This means that a single API call with the payload given will result in the errors: `256 larger than u8::MAX`, `null is not String`, and `extra key "extra" provided`.
47+
48+
This in turn also means that `deer` can be used to implement custom validation while deserializing, while still being able to return all validation issues.
49+
50+
<sub>
51+
52+
`deer` might provide a way in the future to describe these constraints.
53+
54+
</sub>
55+
56+
## Limitations
57+
58+
`deer` currently does **not** parse values itself, but relies on external parsers like `serde_json`, this means that parsing will be fail-fast, and `deer` only touches syntactically correct values.
59+
60+
## Future Plans
61+
62+
The first release of `deer` is intentionally minimal and tries to lay a good foundation to extend functionality in the future. There are many future possible directions and ideas we're trying to see if they can benefit the different use cases.
63+
64+
### Introspection Support
65+
66+
Currently, popular crates like `serde` do not provide a way to introspect what the output will be. Other tools try to fill the gap by manually interpreting the instructions given to these crates. This often leads to edge cases, resulting in the dissonance between the expected value and reality. The goal with the explicit support of introspection is to allow other tools to make use of it and build abstractions around it instead of trying to reverse engineer.
67+
68+
<sup>
69+
How the introspected format might look like is currently unknown.
70+
</sup>
71+
72+
### Validation
73+
74+
Currently, to be able to do validation, one must create a new type that performs the validation step. This extra boilerplate is often a pain point. The idea is to instead allow for _optional_ validation via combinators using the derive macro.
75+
76+
<details>
77+
<summary>What it may look like</summary>
78+
79+
```rust
80+
#[derive(deer::Deserialize)]
81+
struct Payload {
82+
#[validate(all(min(12), max(24)))]
83+
length: u8
84+
}
85+
```
86+
87+
</details>
88+
89+
### Lax Deserialization
90+
91+
Instead of strictly deserializing types, one might prefer to deserialize while coercing values (`"1"` might be interpreted as `1` instead). This behavior would be opt-in instead of opt-out and enabled on the `derive` level.
92+
93+
### Deserializer with Parser
94+
95+
`deer` currently relies on external tools (notably `serde`) to implement parsing of different formats, which means that we still fail fast during the parsing of malformed input. In the future, `deer` might provide a parser that tries to recover from parsing errors and provide meaningful diagnostics.
96+
97+
<details>
98+
<summary>What it may look like</summary>
99+
100+
```text
101+
{
102+
"i8": "string"
103+
```
104+
105+
</details>
106+
107+
### Level of "`deer`"
108+
109+
`deer` strives to be as composable and configurable as possible, which means that all non-core behavior should be opt-in, and the speed penalty of bare `deer` should be minimal. In the future, we might want to provide additional configuration parameters (maximum depth, the maximum number of errors, etc.) to allow for further tweaking in all use cases where speed is of utmost importance.
110+
111+
## Contributors
112+
113+
`deer` was created by [Bilal Mahmoud](https://github.com/indietyp). It is being developed in conjunction with [HASH](https://hash.dev/). As an open-source project, we gratefully accept external contributions and have published a [contributing guide](https://github.com/hashintel/labs/blob/main/.github/CONTRIBUTING.md) that outlines the process. If you have questions, please create a [discussion](https://github.com/orgs/hashintel/discussions). You can also report bugs [directly on the GitHub repo](https://github.com/hashintel/labs/issues/new/choose).
114+
115+
## License
116+
117+
`deer` is available under either of the [Apache License, Version 2.0] or [MIT license] at your option. Please see the [LICENSE] file to review your options.

0 commit comments

Comments
 (0)