Changing dmd to implement `in` meaning `scope const` when DIP1000 is
selected causes Phobos to fail to compile. This changes Phobos in
preparation for that change.
Fix issue 19297: handle comparison of signed, unsigned and floating JSONValues as in D
merged-on-behalf-of: Vladimir Panteleev <github@thecybershadow.net>
Rewrite std.json.JSON_TYPE to be CamelCase with lowercase members, with underscore attached to avoid keyword collision.
Deprecate the old type via alias.
The test corpus provided at https://github.com/nst/JSONTestSuite/ revealed some issues with the std.json.parseJSON function. Since addressing some of the issues required parseJSON to reject input it previously accepted, I have added a new JSONOptions.strictParsing flag so callers can opt-in to the stricter parsing.
The issues, and how I've addressed them, are listed below (approximately from most severe to least):
Silently dropping ASCII NUL characters from strings:
n_string_unescaped_crtl_char.json
This is the most serious problem I found while fixing the test cases. The current implementation of parseJSON() uses a helper function called peekChar() which can store the next character to handle in a variable of type Char (an alias of the character type). Unfortunately it was using 0 to indicate it has not read a character yet so if an ASCII NUL (which will have the value 0) is present in the text and someone reads it with peekChar() then it will effectively be skipped over, which was happening in string and whitespace parsing.
I changed peekChar() to use a Nullable!Char as the temporary storage for the next character to disambiguate the case where there is no pending unconsumed character from the case where there is a pending unconsumed ASCII NUL. In strict mode JSON with unescaped ASCII NULs in strings will throw an exception while in non-strict mode the JSON will be accepted with the NUL included in the string value.
Failure to accept ASCII DEL (0x7f) unescaped in strings:
y_string_unescaped_char_delete.json
y_string_with_del_character.json
These were the only test cases that std.json rejected that it should have accepted. This issue was addressed by changing the string parsing logic to explicitly check for character values < 0x20 instead of using std.ascii.isControl (which also returned true for 0x7f), with a special exception for ASCII NULs in non-strict mode as mentioned above.
Parsing "true", "false", and "null" tokens case-insensitively:
n_structure_capitalized_True.json
In strict mode those tokens are now parsed case-sensitively.
Accepting control characters other than ' ', '\t', '\r', and '\n' as whitespace:
n_structure_null-byte-outside-string.json
n_structure_whitespace_formfeed.json
In strict mode only the listed characters are accepted as whitespace, while non-strict mode continues to use std.ascii.isWhite with an additional exception for ASCII NUL for a similar reason as the n_string_unescaped_ctrl_char.json case (the skipWhitespace() function used peekChar() so it didn't handle ASCII NULs consistently; non-strict mode after my changes is actually more permissive than the previous behavior but it is at least consistently permissive).
Silently accepting empty data:
n_structure_no_data.json
In strict mode an exception is now thrown instead of returning an empty value.
Failure to enforce that numbers beginning with 0 cannot have any additional digits in the non-fractional part:
n_number_-01.json
n_number_neg_int_starting_with_zero.json
n_number_with_leading_zero.json
An additional check is now performed in strict mode when the whole part of a number begins with zero to ensure trailing digits are not present.
Failure to check for trailing characters after parsing:
n_array_comma_after_close.json
n_array_extra_close.json
n_multidigit_number_then_00.json
n_object_trailing_comment.json
n_object_trailing_comment_open.json
n_object_trailing_comment_slash_open_incomplete.json
n_object_trailing_comment_slash_open.json
n_object_with_trailing_garbage.json
n_string_with_trailing_garbage.json
n_structure_array_trailing_garbage.json
n_structure_array_with_extra_array_close.json
n_structure_close_unopened_array.json
n_structure_double_array.json
n_structure_number_with_trailing_garbage.json
n_structure_object_followed_by_closing_object.json
n_structure_object_with_trailing_garbage.json
n_structure_trailing_#.json
An additional check is now performed in strict mode to ensure any trailing characters after the initial JSON value are only whitespace.
In addition to the above issues, parseJSON() will throw ConvException for numbers out of the range of double/long/ulong which was not previously documented. I have updated the ddoc comment to reference that exception.
Otherwise generated documentation contains hml source pieces like:
"json.html#.JSONException">std.json.JSONException if the depth exceeds the max depth."
see: https://dlang.org/phobos/std_json.html#.parseJSON.2
