Rework `process_at_option()`

[GB609]

This function is coming straight from coding hell.
Its main logic consists of one illegible long regular expression.

The regex should be split up, even if that means that some of the code has to be duplicated.

[jbrubake]

Looking at this, I think the @option syntax is too restrictive. It doesn't allow -option and it requires pipes to separate options instead of allowing a comma instead (which would be more appropriate when documenting a list of related but not identical options).

What if you changed the hairy regex to something like /^(((-[^[:blank:]]*[[:blank:]]*)|(,[[:blank:]]*)|(\|[[:blank:]]*))+)(.*)$/ and then did some post-processing of commas just like is done with pipes? This regex includes any of these in "term":

• /-[^ ]* */
• /, */
• /\| */

This was a first pass at a new regex so it might need tightening or simplification.

@ shdoc:447 @ function render_docblock_list(docblock, docblock_name, title, nesting) {
 # @set dockblock["option"] A docblock for correctly formated options.
 # @set dockblock["option-bad"] A docblock for badly formated options.
 function process_at_option(text) {

     # Test if @arg is a command-line option description (starting by - or --).
-    if(match(text, /^(((-[[:alnum:]]([[:blank:]]*<[^>]+>)?|--[[:alnum:]][[:alnum:]-]*((=|[[:blank:]]+)<[^>]+>)?)([[:blank:]]*\|?[[:blank:]]+))+)([^[:blank:]|<-].*)?$/, contents)) {
+    if(match(text, /^(((-[^[:blank:]]*[[:blank:]]*)|(,[[:blank:]]*)|(\|[[:blank:]]*))+)(.*)$/, contents)) {
         debug(" → → found correctly formated @option.")

         # Fetch matched values.
         term = trim(contents[1])
-        option_description["definition"] = trim(contents[8])
+        option_description["definition"] = trim(contents[6])

         # Trim spaces around pipes.
         gsub(/[[:blank:]]+\|[[:blank:]]+/, " | ", term)
@ shdoc:560 @ function render_docblock(func_name, description, docblock, nesting) {
                 # Add strong around options, but exclude pipes.
                 term = render("strong", docblock["option"][i]["term"])
                 gsub(/[[:blank:]]+\|[[:blank:]]+/, "** | **", term)
+                gsub(/[[:blank:]]*,[[:blank:]]*/, "**, **", term)
                 # Escape < an >.
                 gsub(/</, "\\<", term)
                 gsub(/>/, "\\>", term)

As an aside, the current rendering requires spaces around the pipes which might be too restrictive. My comma rendering added a single trailing space but doesn't require the comma to be surrounded by spaces initially.

[GB609]

I'll need a bit of time to look into that in detail. I think the restriction on spaces is there to separate the option description from its name.

But to be honest, i'm questioning whether shdoc really needs to apply that much logic here at all.

Why don't we just do:

# @option ([[^:space:]].*) (: .*)

--> At least one none-space as option name (but take everything up to the first colon), and an optional description after a colon.

• First group: whatever shall appear as option name
• if ':' is given, use the second wildcard group as description.
• Maybe we could also consider a way to mask/escape colon for folks which prefer to use punctuation as option names...

This is much more flexible and if the user feels the need to document long and short options in one line, they are free to do so.
I find this confusing, however, because the way long options are used isn't all that uniform to begin with, and it often differs from short options anyway. It starts becoming problematic when the options require additional parameters:

• short: some tools like cut use -f1 (no spaces), others require spaces (like e.g. find -t d)
• long: sometimes tools take them in the form --long=value, others use --long value, others take both.
• sometimes the same option can be used with and without parameter.

Why bother with all that complexity? I'd leave the format here up to the user completely. Shdoc will never be able to cover all different styles and punctuations users want to have. And matching with restricted expressions only means that a lot of 'ill-formatted' documentation will just be dropped altogether which doesn't make shdoc very reliable.

[jbrubake]

I like the idea of a simplistic @option <option list> : <description format. I had thought about using ":" as a delimiter but decided against it because @arg doesn't require it so that would be inconsistent. Without looking at the code that could probably be overcome by allowing, but for backwards compatibility not requiring, a ":" in @arg statements

Using this format would allow expanding @option to something that also handles subcommands without much extra work. Potentially no extra work if @option is renamed to something generic

I would still provide some nice formatting for common cases like escaping "<" and ">" and not bolding separators like pipes and commas but limit that to things that could be easily documented so users know what they are going to get