This new feature makes it possible to expand variables with
dots or other unusual characters in their names.
* README: Update.
* wordsplit.3: Document the use of the ws_namechar member.
* wordsplit.c (is_name_char): New static function.
(ISVARCHR): Removed. Use is_name_char instead.
(_wsplt_seterr): Set errno to EINVAL if WRDSE_USAGE is returned.
(_wsplt_subsplit): Clear the WRDSO_MAXWORDS option in the subsplit.
Pass ws_namechar.
(wordsplit_init): Check for valid ws_namechar content.
(expvar): Take into account ws_namechar when scanning variable name.
Fix name length passed to the _wsplt_setctxerr call.
* wordsplit.h (ws_namechar): New member.
(WRDSO_NAMECHAR): New option bit.
* wsp.c: New options: -D to define an "extra" environment entry
(possibly containing characters not allowed by the shell), and
-namechar to define additional variable name constituents.
* wordsplit.at: Test namechar modifications.
* src/wordsplit.h (WRDSO_GETVARPREF): New option.
* src/wordsplit.c (wordsplit_find_env): Rewrite as wsplt_env_lookup
wsplt_env_getvar): New function.
(expvar): Select preference of wsplt_env_lookup vs. wsplt_env_getvar
depending on the value if WRDSO_GETVARPREF option.
* src/wordsplit.c (wordsplit_run): Initialize ws_errctx early. This
ensures that wordsplit_strerror gets correct data after WRDSE_USAGE.
(wordsplit_free): Return immediately if errno was WRDSE_USAGE.
* include/wordsplit.h (WRDSO_NOVARSPLIT)
(WRDSO_NOCMDSPLIT): New options.
* src/wordsplit.c (scan_word): Treat any variable reference, even
containing whitespace, as a single word if WRDSO_NOVARSPLIT is set.
Ditto for commands and WRDSO_NOCMDSPLIT.
* tests/wordsplit.at: Add new tests.
* tests/wsp.c: Recognize novarsplit and nocmdsplit
options.
For future use: recognize bskeep_words, bskeep_quote, bskeep.
Positional variables ($N and ${N}) are recognized. Variable names in
curly braces follow the same rules as unadorned ones.
This commit also changes memory reallocation strategy in wsplt_assign_var.
If ws_envbuf needs to be expanded, new allocation size is selected as
3/2 of the previous allocation, if that size is less than max(size_t).
This feature allows the user to limit the number of words returned
by a call to wordsplit. When the number of words in expansion reaches
the predefined limit, the rest of input line will be expanded and returned
as a single last word.
For example, to parse a /etc/passwd line:
struct wordsplit ws;
ws.ws_delim = ":";
ws.ws_maxwords = 7;
ws.ws_options = WRDSO_MAXWORDS;
wordsplit(str, &ws, WRDSF_NOVAR | WRDSF_NOCMD | WRDSF_DELIM | WRDSF_OPTIONS);
* doc/wordsplit.3: Document the maxwords feature.
* include/wordsplit.h (wordsplit) <ws_maxwords>
<ws_wordi>: New members.
(WRDSO_MAXWORDS): New option.
* src/wordsplit.c (WSP_RETURN_DELIMS): New macro.
(_wsplt_subsplit): Rewrite.
(wordsplit_init0): Don't reset node list.
(wordsplit_init): Initialize ws_wordi and the node list.
(wsnode_insert): Correctly insert lists.
(coalesce_segment): Additional safety check.
(wsnode_tail_coalesce): New static function.
(wordsplit_finish): Postprocess delimiters.
(expvar,expcmd): Use new _wsplt_subsplit.
(wordsplit_varexp): Don't try to expand delimiter nodes.
(skip_delim): Remove delimiter processing. It is now done
in wordsplit_finish.
(scan_word): New argument 'consume_all' instructs it to
consume the rest of input as one token.
(wordsplit_process_list): Handle wsp->ws_maxwords setting.
This also fixed a long-standing bug: quotes weren't processed
in WRDSF_NOSPLIT mode. See the testcase 59 (incremental nosplit).
(wordsplit_run): Rewrite.
(wordsplit_free): Free node list.
* tests/wordsplit.at: Update for the new wsp output format.
(incremental nosplit): Expect correct output.
Add tests for the maxwords feature.
* tests/wsp.c (maxwords): New flag.
Print the ws_wordi value as "TOTALS" at the end of each run.
This includes the following commits pushed between
2015-09-19 and 2017-10-10:
090c7b9a Allow ws_getvar to set value to NULL and
return MU_WRDSE_OK.
The value is processed as if it were "", i.e.
MU_WRDSE_UNDEF is returned.
64313fdf Fix MU_WRDSF_INCREMENTAL | MU_WRDSF_NOSPLIT
46d7640f Add wordsplit_append function
151eb4b9 Fix nested expansions and command expansions
occurring after variable expansions.
ad3cc340 Replace void wordsplit_getwords with
int wordsplit_get_words.
* include/wordsplit.h (wordsplit_get_words): New function.
(wordsplit_getwords): Mark as deprecated.
(wordsplit_append): New function.
* src/wordsplit.c (wordsplit_append): New function. MU 46d7640f.
(expvar): Treat NULL value as "". MU 090c7b9a.
(expcmd): Allow command and variable expansions in subsplit.
(exptab): Change ordering of expansions so that command expansion
occurs first. This fixes nested expansions and command expansions
occurring after variable expansions. MU 151eb4b9.
(wordsplit_process_list): Update wsp->ws_endp in nosplit mode. This
fixes wordsplit MU_WRDSF_INCREMENTAL | MU_WRDSF_NOSPLIT. MU 64313fdf.
(wordsplit_get_words): New function. MU ad3cc340.
* tests/wordsplit.at: Test the above changes.
* tests/wsp.c: Accept extra arguments to append using wordsplit_append.
* src/wordsplit.c (_WSNF_DELIM): New node flag
(wsnode_flagstr): Print 'd' for that flag.
(expvar): Don't set WRDSF_WS when expanding variable
value. This fixes, e.g., expansion of a$x if x begins with
a whitespace.
(wsnode_nullelim): A _WSNF_DELIM node breaks join chain.
(wordsplit_trimws): Don't affect nodes within join chains.
(scan_word): Set _WSNF_DELIM for returned delimiters.
* tests/wordsplit.at (TESTWSP): 8th argument - prologue
shell code. Use it to unset environment variables in
env/getenv tests.
Test ws elimination with return_delims set.
* src/wordsplit.c: Implement default assignment, word
expansion in variable defaults, distinction between
${variable:-word} and ${variable-word}.
* doc/wordsplit.3: New file.
* src/wordsplit.h (wordsplit)<ws_envbuf,ws_envidx>
<ws_envsiz>: New members.
(WRDSF_ARGV): Remove.
(WRDSF_OPTIONS): New flag.
(WRDSO_ARGV): New option bit.
* tests/wordsplit.at: Add new tests.
* tests/wsp.c: Set WRDSF_OPTIONS flag if one of the options is requested.
