MediaType::toString can produce unparsable results

[mizosoft]

I've found cases where MediaType::toString generates erroneous results.

One case is when the MediaType instance has a parameter of an empty value. This can happen when calling withParameter("foo", ""), or parsing text/plain; foo="". When the parameters are later joined, the value is erroneously matched as a token instead of being quoted (as tokens cannot be empty). This will lead the following to throw IllegalArgumentException:

MediaType type1 = MediaType.parse("text/plain; foo=\"\"");
MediaType type2 = MediaType.parse(type1.toString()); // Trying to parse 'text/plain; foo=' will fail

I think this line should first check if the string is empty before matching against TOKEN_MATCHER, or maybe just return the literal "\"\"" on that case to avoid the StringBuilder allocation.

Another case is when parameters are set explicitly using withParameter[s](...), where parameter value[s] contain non-ASCII characters. For example this will also throw IllegalArgumentException:

MediaType type1 = MediaType.create("text", "plain").withParameter("let_me_in!", "\"جوافة\"");
MediaType type2 = MediaType.parse(type1.toString()); // The tokenizer will fail to consume QUOTED_TEXT_MATCHER

I don't know whether this is the intended behaviour or not but MediaType::toString can be used in HTTP header values which shouldn't contain non-ASCII octets. IMHO normalizeParameterValue(...) should validate the value against the ascii() matcher.

[cpovirk]

Thanks, that all makes sense. I see the part that requires tokens to be nonempty in https://tools.ietf.org/html/rfc2045#section-5.1 and in our parse implementation.

And I see that quoted strings can be empty: https://tools.ietf.org/html/rfc2822#section-3.2.5

Requiring ASCII makes sense in light of https://www.ietf.org/rfc/rfc2045.txt and the existing parse checks.

[cpovirk]

Assuming that this doesn't produce breakages in some internal tests, I'll see if I can get it fixed shortly. If it does produce breakages, then it may be longer.

[mizosoft]

Hi, thanks for fixing. Sorry for not including the relevent RFC.
Unfortunately, it appears that i forgot to point out that empty tokens can also pass through when explicitly set, for example these calls should all fail:

MediaType.create("text", "");
MediaType.create("", "plain");
MediaType.create("text", "plain").withParameter("", "i don't have a name");

I think adding && !token.isEmpty() here would do.
Sorry for not being able to point that out before the issue was closed.