* include/wordsplit.h (WRDSO_NOVARSPLIT)
(WRDSO_NOCMDSPLIT): New options.
* src/wordsplit.c (scan_word): Treat any variable reference, even
containing whitespace, as a single word if WRDSO_NOVARSPLIT is set.
Ditto for commands and WRDSO_NOCMDSPLIT.
* tests/wordsplit.at: Add new tests.
* tests/wsp.c: Recognize novarsplit and nocmdsplit
options.
For future use: recognize bskeep_words, bskeep_quote, bskeep.
Positional variables ($N and ${N}) are recognized. Variable names in
curly braces follow the same rules as unadorned ones.
This commit also changes memory reallocation strategy in wsplt_assign_var.
If ws_envbuf needs to be expanded, new allocation size is selected as
3/2 of the previous allocation, if that size is less than max(size_t).
This feature allows the user to limit the number of words returned
by a call to wordsplit. When the number of words in expansion reaches
the predefined limit, the rest of input line will be expanded and returned
as a single last word.
For example, to parse a /etc/passwd line:
struct wordsplit ws;
ws.ws_delim = ":";
ws.ws_maxwords = 7;
ws.ws_options = WRDSO_MAXWORDS;
wordsplit(str, &ws, WRDSF_NOVAR | WRDSF_NOCMD | WRDSF_DELIM | WRDSF_OPTIONS);
* doc/wordsplit.3: Document the maxwords feature.
* include/wordsplit.h (wordsplit) <ws_maxwords>
<ws_wordi>: New members.
(WRDSO_MAXWORDS): New option.
* src/wordsplit.c (WSP_RETURN_DELIMS): New macro.
(_wsplt_subsplit): Rewrite.
(wordsplit_init0): Don't reset node list.
(wordsplit_init): Initialize ws_wordi and the node list.
(wsnode_insert): Correctly insert lists.
(coalesce_segment): Additional safety check.
(wsnode_tail_coalesce): New static function.
(wordsplit_finish): Postprocess delimiters.
(expvar,expcmd): Use new _wsplt_subsplit.
(wordsplit_varexp): Don't try to expand delimiter nodes.
(skip_delim): Remove delimiter processing. It is now done
in wordsplit_finish.
(scan_word): New argument 'consume_all' instructs it to
consume the rest of input as one token.
(wordsplit_process_list): Handle wsp->ws_maxwords setting.
This also fixed a long-standing bug: quotes weren't processed
in WRDSF_NOSPLIT mode. See the testcase 59 (incremental nosplit).
(wordsplit_run): Rewrite.
(wordsplit_free): Free node list.
* tests/wordsplit.at: Update for the new wsp output format.
(incremental nosplit): Expect correct output.
Add tests for the maxwords feature.
* tests/wsp.c (maxwords): New flag.
Print the ws_wordi value as "TOTALS" at the end of each run.
This includes the following commits pushed between
2015-09-19 and 2017-10-10:
090c7b9a Allow ws_getvar to set value to NULL and
return MU_WRDSE_OK.
The value is processed as if it were "", i.e.
MU_WRDSE_UNDEF is returned.
64313fdf Fix MU_WRDSF_INCREMENTAL | MU_WRDSF_NOSPLIT
46d7640f Add wordsplit_append function
151eb4b9 Fix nested expansions and command expansions
occurring after variable expansions.
ad3cc340 Replace void wordsplit_getwords with
int wordsplit_get_words.
* include/wordsplit.h (wordsplit_get_words): New function.
(wordsplit_getwords): Mark as deprecated.
(wordsplit_append): New function.
* src/wordsplit.c (wordsplit_append): New function. MU 46d7640f.
(expvar): Treat NULL value as "". MU 090c7b9a.
(expcmd): Allow command and variable expansions in subsplit.
(exptab): Change ordering of expansions so that command expansion
occurs first. This fixes nested expansions and command expansions
occurring after variable expansions. MU 151eb4b9.
(wordsplit_process_list): Update wsp->ws_endp in nosplit mode. This
fixes wordsplit MU_WRDSF_INCREMENTAL | MU_WRDSF_NOSPLIT. MU 64313fdf.
(wordsplit_get_words): New function. MU ad3cc340.
* tests/wordsplit.at: Test the above changes.
* tests/wsp.c: Accept extra arguments to append using wordsplit_append.
* src/wordsplit.c (_WSNF_DELIM): New node flag
(wsnode_flagstr): Print 'd' for that flag.
(expvar): Don't set WRDSF_WS when expanding variable
value. This fixes, e.g., expansion of a$x if x begins with
a whitespace.
(wsnode_nullelim): A _WSNF_DELIM node breaks join chain.
(wordsplit_trimws): Don't affect nodes within join chains.
(scan_word): Set _WSNF_DELIM for returned delimiters.
* tests/wordsplit.at (TESTWSP): 8th argument - prologue
shell code. Use it to unset environment variables in
env/getenv tests.
Test ws elimination with return_delims set.
* src/wordsplit.c: Implement default assignment, word
expansion in variable defaults, distinction between
${variable:-word} and ${variable-word}.
* doc/wordsplit.3: New file.
* src/wordsplit.h (wordsplit)<ws_envbuf,ws_envidx>
<ws_envsiz>: New members.
(WRDSF_ARGV): Remove.
(WRDSF_OPTIONS): New flag.
(WRDSO_ARGV): New option bit.
* tests/wordsplit.at: Add new tests.
* tests/wsp.c: Set WRDSF_OPTIONS flag if one of the options is requested.
* src/wordsplit.c (wordsplit_tildexpand)
(wordsplit_pathexpand): New functions.
(wordsplit_process_list): Run tilde and pathname expansion
if WRDSF_PATHEXPAND flag is set.
* src/wordsplit.h (wordsplit)<ws_options>: New member.
(WRDSF_PATHEXPAND): New flag.
(WRDSO_NULLGLOB,WRDSO_FAILGLOB)
(WRDSO_DOTGLOB): New defines.
* tests/wsp.c: New options pathexpand, nullglob, failglob,
dotglob. Fix help output.
* src/wordsplit.c (_wsplt_seterr): New static function. Use instead
of assigning value to ws_errno.
(coalesce_segment): Propagate _WSNF_QUOTE flag to the resulting node.
(wordsplit_process_list): Coalesce the list before running
command expansion.
* tests/testsuite.at: Test wordsplit first. If it fails, most other
tests will fail too.
* tests/wordsplit.at: Test command expansion.
New invocation sequence ensures proper error handling.
This is an incompatible change. Authors using ws_getvar member will
have to rewrite their ws_getvar function accordingly.
* src/wordsplit.c (wordsplit_init0): Call wordsplit_clearerr
on reuse.
(wordsplit_init): Fix ws_errno
(expvar): Change invocation of ws_getvar.
(expcmd): Change invocation of ws_command.
(wordsplit_clearerr): New function.
(wordsplit_strerror): Handle WRDSE_USERERR.
* src/wordsplit.h (ws_getvar): Change return value and signature of
ws_getvar and ws_command.
New member 'ws_usererr'.
(WRDSF_ARGV): New flag.
(WRDSE_OK): New define. Same as WRDSE_EOF.
(WRDSE_USERERR): New error code.
(wordsplit_clearerr): New proto.
* tests/wsp.c (wsp_getvar, wsp_runcmd): Rewrite.
* src/wordsplit.c (wordsplit_init): Change handling of lacking
WRDSF_NOCMD.
(find_closing_cbrace): Rename to find_closing_paren, take additional
argument. All uses changed.
(node_expand_vars): Rewrite as a generalized function node_expand.
(wordsplit_varexp): Use node_expand.
(expcmd, wordsplit_cmdexp): New functions.
(scan_word): Handle $(command) notation
(wordsplit_process_list): Command expansion.
* src/wordsplit.h (wordsplit) <ws_command>: New member.
* tests/wsp.c (wsp_runcmd): New function.
(main): Set ws_command unless WRDSF_NOCMD flag is set.
* tests/wsp.c (help): Update.
(wsp_getvar): New function.
(main): Callect all arguments in form X=Y to an array
and set ws_getvar member if this array is not empty.
* tests/wordsplit.at: Add tests for ws_getenv function.
* src/wordsplit.c (expvar): Ensure value is set.
Pass WRDSF_QUOTE to wordsplit
(scan_word): Treat ${...} as a single word.
* tests/wordsplit.at: Add more tests.
* tests/wsp.c: Fix error message.
* src/wordsplit.c (ISVARBEG,ISVARCHR): New macros.
(expvar): ws_getvar allocates memory.
* src/wordsplit.h (wordsplit)<ws_getvar>: Remove const
from the return value: the function should allocate memory.
