Windows stdio handles can be null

[ChrisDenton]

See also: rust-lang/rust#88576

In short, Windows stdio functions can return null to mean that there are no handles. The code here assumes stdio handles are always valid File handles. While this is technically an incorrect assumption, it worked out ok until Rust's new "Safe I/O" code was implemented.

A workaround will be implemented upstream but I thought there should be an issue here too, since unfortunately it reached stable, causing env_logger to break.

[BurntSushi]

Yeah, this impacts termcolor too: BurntSushi/termcolor#53

When stdio handles are null on Windows, what does println! do?

[ChrisDenton]

Nothing appears on screen but it's successful. And stdout().write reports the bytes as written. I'm actually a bit surprised by that because I would have thought it was just calling the WriteFile API on the handle (which should fail). I wonder what it's writing to...

[BurntSushi]

That's really weird. Also, I would have expected println! to panic and writeln! to return an error given that it has a null stdout handle? Like, how does io::stdout() even work given that it's null? It seems like our std APIs here are presenting a lie...

[ChrisDenton]

Yeah, it really did surprise me. I had to check I wasn't misunderstanding something. From a quick investigation it seems like stdio swallows ebadf errors (I'd assume "ebadf" is a nix-ism) and pretends they were successful. On Windows is_ebadf returns true for INVALID_HANDLE_VALUE and elsewhere null is mapped to INVALID_HANDLE_VALUE.

So, yeah, the std APIs lie.

[ChrisDenton]

Ok, I'm told this is very much intentional because otherwise, for example, println! would panic when running a program that's detached from consoles (e.g. a Windows GUI program). So the stdio APIs are meant to silently fail on bad values whereas File is free to assumes all handles are valid (because that's the promise its API makes and is meant to be upheld).

[BurntSushi]

Silent failures don't seem great. :-/ println! already panics when stdout gets closed anyway.

But yeah, not quite sure what to do here.

[ChrisDenton]

Hm, I think this needs some serious design work. Obviously changing the standard library behaviour breaks programs so some new APIs might be required. I think I'd need to understand more about how this is being used in practice.

One simple thing that could be done is for Stdout, etc to have a try_as_handle. That doesn't solve the problem but at least the API indicates that there is an issue to be aware of. Maybe a higher level API would be try_as_file_ref. I'm not sure. And I probably shouldn't be throwing out random ideas before I've had time to consider them more. 🤷