Tracking issue for concrete errors in the standard library

[alexcrichton]

This is a tracking issue for the unstable str_parse_error and utf8_error features in the standard library. Some various small error types popped up once FromStr had an associated error type and haven't been considered too closely for stabilization.

Some thoughts are:

• Are the names of these types appropriate?
• Are we covered in terms of future extensibility?
• Are the relevant portions of the error exposed well?

[aturon]

Nominating for 1.5 FCP discussion.

[BurntSushi]

The str_parse_error feature seems to be for this code:

/// Error returned from `String::from`
#[unstable(feature = "str_parse_error", reason = "may want to be replaced with \
                                                  Void if it ever exists",
           issue = "27734")]
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
pub struct ParseError(());

#[stable(feature = "rust1", since = "1.0.0")]
impl FromStr for String {
    type Err = ParseError;
    #[inline]
    fn from_str(s: &str) -> Result<String, ParseError> {
        Ok(String::from(s))
    }
}

This does indeed seem to be a good candidate for a void type. However, it is actually a unit type. Was that intended?

For utf8_error, it looks like it's just the valid_up_to method, which seems pretty reasonable to me.

[SimonSapin]

For utf8_error, it looks like it's just the valid_up_to method, which seems pretty reasonable to me.

The currently documented semantics are not very helpful:

Starting at the index provided, but not necessarily at it precisely, an invalid UTF-8 encoding sequence was found.

I’d like to know precisely.

Taking a step back, I’d like to reproduce the functionality of String::from_utf8_lossy, but without allocating memory / copying data. Emitting a stream of &str slices and '\u{FFFD}' replacement characters, but have the correct number of replacement characters. Could enough information be added on Utf8Error to make this doable by calling str::from_utf8 repeatedly? Or should this be a new API entirely?

[SimonSapin]

Never mind. For the use case I’m thinking of I also want to read the input bytes in chunks, and support decoding a UTF-8 byte sequence for a single code point split across chunks. I’ll probably build it in an external library and then propose for eventual inclusion in libcore, where it could (I hope) share code with str::from_utf8.

Still, the "not necessarily at it precisely" part of valid_up_to is not very satisfying. Perhaps it could be documented as "the maximum index such that str::from_utf8(input_bytes[..index]) returns Some(_)" ? If I’m reading the from_utf8 code correctly, that’s what it is already.

Could Utf8Error even have a method that returns that prefix &str slice, rather than just an index? Or would adding the lifetime parameter be a breaking change to stable API?

[alexcrichton]

This issue is now entering its cycle-long FCP for stabilization in 1.5

---

@SimonSapin yeah one thing we're particularly interested in are what exact guarantees we can provide in from the Utf8Error type, especially around the indexes returned. I believe that adding a lifetime parameter would be backwards-incompatible now, so that's probably out of the question. Beyond that though, it sounds like you think the wording around the documentation should be tweaked, and perhaps another piece of information could be added along the lines of "this is the byte where something bad happened"? Or perhaps also a "skip to this position to keep decoding" kinda position?

Curious to hear your thoughts!

[SimonSapin]

I think that Utf8Error::valid_up_to can be stabilized with #28637

It should be backward-compatible to add more methods (and private fields) later to provide more info about the error and how to decode the rest of the input. But I’m not 100% sure yet what that should look like. For example, if the input ends with 1~3 bytes that look like the beginning of a valid but incomplete UTF-8 sequence, some users might want re-assemble that code point if more input is coming later.

[alexcrichton]

Ok cool, thanks @SimonSapin! I'd be totally down for adding more information later if we can!