Replace block module with helper functions

[newpavlov]

This change allows to remove the serde feature and reduce total amount of code in the crate. Additionally, the helper block functions now store the cursor position in the first buffer value, which allows us to remove the separate index field.

The new examples demonstrate implementation of the traits for the most common RNG classes.

TODO:

• Check value stability in downstream impls.
• Write integration tests with 100% coverage.

[dhardy]

What does this look like at the use site? You could update rand_chacha at least (same repo; yes I am going to have to remake those PRs to move it to the rngs repo anyway).

[dhardy]

Motivation regarding serde is good; the feature does not really belong in this crate.

lazy block generation

Not a change.

The le module is renamed to utils

I explained why I didn't want to do this in rust-random/rand#1667.

[newpavlov]

What does this look like at the use site?

See the new doc examples. I wanted to update the implementation crates after I got a confirmation that you approve the new approach.

Not a change.

True. I will edit the OP.

I explained why I didn't want to do this

The problem is that the new_buffer function and the Word trait technically have nothing to do with "litttle endian". And I think utils::fill_bytes_via_gen_block looks better in user code than le::fill_bytes_via_gen_block. But it's not a strong opinion, so I can rename it back, if you insist.

[newpavlov]

All potential panics in fill_bytes_via_gen_block get eliminated successfully: https://rust.godbolt.org/z/6E9Y7vW3o Unfortunately, codegen is a bit worse than expected. The compiler unrolls the head and tail processing loops instead of compiling them into one memcpy. I will look into whether it's possible to tweak the code to prevent it.

next_word_via_gen_block looks good: https://rust.godbolt.org/z/5qd7MaKx3 Same for fill_bytes_via_next_word: https://rust.godbolt.org/z/esYo49oKK

[newpavlov]

I improved codegen for fill_bytes_via_gen_block (see https://rust.godbolt.org/z/T8f77KjGc), but unfortunately it involves unsafe code. In future it could be removed with generic_const_exprs, but for now I couldn't find a way to do it on Rust 1.85 without returning to the bad codegen.

[dhardy]

I don't want to merge this for two reasons:

1. Performance issues affecting u32 / u64 generation from rand::rng() (see comments on rand#1686).
2. The results buffer has poor type safety (see above and on rand#1686).

Some changes are probably worth keeping. #29 includes some of the unproblematic changes.

[newpavlov]

I replaced the utility functions with the new BlockBuffer type which does the same thing, but enforces the buffer invariant.

I will work on ReseedingRng performance next.

[dhardy]

As I said here:

I briefly considered adding a Buffer struct, but the result is mostly the same as BlockRng.

[newpavlov]

As I wrote in previous discussions, I think we should completely remove the block RNG trait since its only use in practice is ReseedingRng.

[dhardy]

In practice though you are just using a closure instead of a trait here. It doesn't make for a cleaner interface.

[newpavlov]

This is an implementation detail. Take look at the examples, the code looks pretty clean to me.

This approach also makes a bit easier to implement the serde traits, since users can derive them for core as usual and special handling is only needed for the buffer.

[dhardy]

Looks like a re-write of the block code you wanted to get rid of, minus the traits of course.

This approach also makes a bit easier to implement the serde traits, since users can derive them for core as usual and special handling is only needed for the buffer.

How is that any different from this?

[dhardy]

Comments only (not a change request since I do not plan to merge this).

I gave two reasons for rejecting this PR above, but here's a third: we don't need to rewrite what already exists.

I attempted to ascertain your goals here in #31, and I think we can hit those same goals without a rewrite of what we already have. Which, given that the current code is decently well proven, is quite significant in my opinion.

Take look at the examples, the code looks pretty clean to me.

Neat examples are not very important. Validation against existing code like the rand repository is much more important.

I think we should completely remove the block RNG trait since its only use in practice is ReseedingRng.

Opinionated take. What you have produced here is just a cut-down variation on the existing block module. #34 reduces this to just Generator, CryptoGenerator and BlockRng. Effectively, this code is replacing Generator with a closure (not my preference when the flexibility of closures is not required) and dropping CryptoGenerator.

My view is that we should limit changes to the rand_core API to only things which have clear motivation.

[newpavlov]

we don't need to rewrite what already exists.

IMO it's a wrong stance to have if you intend to release v1.0 in the near future. Your Generate trait is clearly deficient in the sense that it can be improved with more powerful const generics. And as argued before, I am not convinced that we need it in the first place.

Additionally, this PR makes the API surface of rand_core smaller and reduces size of buffered structs.

Validation against existing code like the rand repository is much more important.

What are you talking about? I validated the proposed approach in rust-random/rngs#78 and rust-random/rand#1686. They both show clear reduction of total code size.

The only problem is performance of ReseedingRng (the sole user of your trait). And it can be worked around with a closure-based interface.

If you intend to keep the Generator trait, then I think it should be at the very least moved out of rand_core.

[dhardy]

IMO it's a wrong stance to have if you intend to release v1.0 in the near future.

IMO it's the wrong stance to rewrite everything right before stabilisation.

What are you talking about? I validated the proposed approach in rust-random/rngs#78 and rust-random/rand#1686. They both show clear reduction of total code size.

Yes, that's what I'm talking about. Sorry, I missed your updates for some reason.

The only problem is performance of ReseedingRng (the sole user of your trait). And it can be worked around with a closure-based interface.

If you intend to keep the Generator trait, then I think it should be at the very least moved out of rand_core.

Closures are just another form of interface, and I find them less neat personally. (Excepting where the closures are actually doing interesting things; not just accessing a field.)

Moving Generator out of rand_core would require moving the block code out too. Which we could do.

My intention actually (as mentioned in #13) is to move everything except what we are already happy to stabilise (maybe just RngCore) to a new crate so that we can publish rand_core v1.0 with stable RngCore, and tell people to use the other crate if they need anything more. So a decision about whether or not we want block code in rand_core can be deferred.

Your Generate trait is clearly deficient in the sense that it can be improved with more powerful const generics.

With future Rust features, yes. I don't think we can bet on those landing in the near future (read: next few years).

This is a good argument for keeping the trait out of rand_core, though it's not the only thing blocked by missing Rust features (another is the correct set of impls for TryRngCore).

[newpavlov]

IMO it's the wrong stance to rewrite everything right before stabilisation.

As I wrote before, I would prefer to not stabilize rand_core right away.

Closures are just another form of interface, and I find them less neat personally.

Yes. But my point is that we don't have to make it "neat". The practice shows that users do not care about the block RNG interface. Implementation of buffering should be nothing more than an implementation detail.

The primary reason why we care about exposing the block RNG interface is ThreadRng and we can handle it fine by exposing the block parts in chacha20 with inherent methods and wrapping them using closure-based interface.