Add a macro preprocessor for documentation?

[Luashine]

Please refer to my proof of concept repository, the code is runnable: https://github.com/Luashine/poc-macro-preprocessor

Short quote:

Motivation

I'm growing tired of having roughly equal (often exactly equal) text documentation blocks that must be included in multiple places across jassdoc.
This is an attempt to make a macro language/processor that's pleasant to use.

Goals:

• Make macro preprocessing optional for the writer.
• The entire text document must be as easily readable without the preprocessor active. This means changes must be applied to the file.
• Inline macro definitions
• Macros should be updatable at any point in time
• Easy to use and understand for non-programmers

Macro definition - @start block:

@start <name> <any text including multiline> end@

Macro reusage - @paste block:

@paste <name> <empty or previous macro value> end@

Example

Here it updates / replaces @paste directives.

Running command: 'diff 'macro-test/complex.txt' 'macro-test/complex.txt.out''

29c29
< @paste handle  end@
---
> @paste handle It's a regular handle. end@
35c35,40
< @paste handlemulti  end@
---
> @paste handlemulti It's a really
> really
> ordinary
>
> handle
> ok? end@

---

@lep thoughts?

1. Because continuing with manual copy-pasting will increase burden and inconsistencies later on.
2. Maybe I don't know of some super-duper cool doc/macro tools, but this is simple and tailored to our needs
3. The ingestion will need minor changes to how @note are processed. Only to drop the @start/@paste <name> end@ in them.
   • however it is not a hard requirement, because plain-text is kept in place and readable.
4. Lua or Bash? My goal was to minimize dependencies as well, so just a basic shell with coreutils would be enough.
   • The Bash version may look scarier but is more robust, because it's already based on a state machine and can point out errors early.
   • Lua needs an interpreter installed. Perl too. Although both may be preinstalled on some distros, especially Perl. No to both on Windows, while Bash can be used with coreutils out of the box with Cygwin/git for Windows
   • I don't know awk to use in place of Bash 😛 and I never learned Perl either.
5. Needs to be run manually to apply changes to old/new macros. Otherwise all text remains there

TODO: Finish and polish one of the scripts & change macro blocks to be "macro-new / macro-paste".

EDIT: The macro content is plain text by definition. Initially I intended macros to only be usable at the start of a line, end-tag must be at the end. It may result in a @note in body, but can contain just any regular text.

[lep]

It's an interesting issue for sure. Is there any prior art in any other API-doc system?
jassdoc has an actual jass parser in it so i do now the AST of each thing with a docstring. This mean we could use "scoped" names to refer to stuff, something like this:

/**
@note [^note-1] This is a note
*/
type agent extends handle


/**
@copy agent:note-1
*/
type unit extends agent

Syntax is of course subject to change. But would this fit your view?

[Luashine]

Sorry for a wall of text. You only need to read the first section.

TLDR:

1. No. Other systems have simple or C-like macros, or more complex include directives. None of them feature inline previews for recurring editing and ease of use.
   • this is a first. maybe I need to publish my own paper, typeset in LaTeX to join the giants like Knuth 🙈
2. I don't contend on your suggested syntax, anchoring the definition to a specific function/variable is a good idea.
3. However, I would love to still have text pasted (and updated) inline into "use" blocks.
4. I am afraid installing a Haskell env would be too much for semi-casual editing. I would need to test it first.
5. After all, parameters would be nice to have (may not need inline replacement, because they are within context):

/**
@macro description(X) Returns $1 map coordinate of whichUnit (alive or dead). Returns 0.0 if unit was removed or is null.

@macro zeppelin @bug If the unit is loaded into a zeppelin this will not return the position
of the zeppelin but the last position of the unit before it was loaded into
the zeppelin.

@macro widgettype(`unit`) @note Since $1 extends from `widget`, you can use widget-related functions too.

@patch 1.00
*/
constant native GetUnitX            takes unit whichUnit returns real


/**
@reuse GetUnitX:description(Y) Returns $1 map coordinate of whichUnit (alive or dead). Returns 0.0 if unit was removed or is null.

@reuse GetUnitX:zeppelin @bug If the unit is loaded into a zeppelin this will not return the position
of the zeppelin but the last position of the unit before it was loaded into
the zeppelin.

@reuse GetUnitX:widgettype(`unit`) @note Since $1 extends from `widget`, you can use widget-related functions too.

@patch 1.00
*/
constant native GetUnitY            takes unit whichUnit returns real

Advantages:

1. Immediately clear where to look for definition (however, scope is limited to a single file?)
2. Unnamed arguments shall be enough
3. Body block is inlined, don't need to jump back and forth
4. Naming matters to newbies, hence @macro and @reuse
5. I don't see any additional obstacles for localization, if you consider macro arguments translatable strings too. (distant goal)
6. Downside: requires heavier preprocessing
7. -/+ must adhere to @note like syntax

---

To quote ChatGPT's word salad:

Looking for prior art is a great way to ground your work in existing concepts.

---

Indeed, I haven't looked at prior art before and just kept going at it.

1. Doxygen:
   • copydoc/brief/details command - seems to only affect output and is close to your proposed syntax. not flexible in that Doxygen can only copy all/brief/details section
   • alternative solution: frankenstein of creating pseudo-pages in docs and then linking to them
   • aside: has the concept of grouping (like World Edit categories), something we'll eventually need to incorporate WE categories and to create our own too.
2. Sphynx:
   • has variable substitution via reStructuredText - useful for branding, but not real "variables" that could be used for GetUnitX vs GetUnitY (my simplistic approach doesn't have this either)
   • file includes too
3. Markdown extensions
   • this is a wild west and our "Markdown" is localized each to its own block anyway. We need something overarching, something global. And a separate file with definitions is tedious to work with
   • Static site generators like Hugo have an inline function processing (similar to Shell pipes) like described here
   • Here's specifically Jekyll, where you "jump out" of Markdown context with their syntax to modify Markdown generation
     1. {% assign some_array = "foo,bar,baz" | split: "," %}
     2. Some Array, Original: {{ some_array | inspect }}
   • doesn't fit jassdoc
4. Latex macro aka "newcommand" - explanation
   • similar concept: cmdname = text value, then \cmdname to reuse it
   • supports numbered variables
   • no preview (i.e. no full text insertion), so you'll need to memorize or look it up every time
5. AsciiDoc:
   • include is a very powerful directive to tap into other documents at any fine-grained level. Like specific lines only
   • macros seem to be straight up custom function calls (optionally with parameters) to unfold and substitute a macro. Good example in asciidoctor.js
   • Again it's not writer-focused, only affects output
6. DocBook (XML)
   • relies on XML entity syntax for substitution and parameters defined at meta level - example at SO
   • it's 2025 and this is clunky XML 😂
7. Emacs orgmode macros:
   1. #+MACRO: poem Rose is $1, violet's $2. Life's ordered: Org assists you.
   2. {{{poem(red,blue)}}}
8. ChatGPT was helpful enough to point towards Knuth's WEB (as in spider web, not WWW) doc processor
   • apparently a different programming paradigm, where code and documentation are interwoven into one: https://stackoverflow.com/questions/31331971/what-is-knuths-web
   • page 5
   • @d name(#) == text value syntax, where # is optional syntax for a parameter. includes recursive unfolding of macros/parameters
   • why am I not surprised to see Knuth mentioned? :)
   • again, focused on post-processed output only
   • advanced syntax is too complex

---

Apparently, none of these systems prioritized ease of editing and source file reading. On second thought, this is only a problem, because neither TeX nor jassdoc nor other such systems have WYSIWYG editing capabilities. Only in that case would a writer not care what the source file looked like underneath. Actually, Confluence has this type of editable previews (video, ~30s).

The issue I want to not have is breaking context. When I edit/read a codified doc, I do not want to jump back and forth to lookup a macro. I want it inline and easy to comprehend*. Not just for editing (write-only) but proof-reading too (read-only), whether the entire function's doc is coherent, without a need to look at the fully processed output. The worst contender there would be TeX with its unreadable syntax.

* again, not even "serious" systems made for large text works like AsciiDoc put the author first, imo. It would be okay to have some master document of a book to look like this:

# Introduction

include::intro.adoc[]

# Chapter 1: The beginning

include::ch01.adoc[]

etc. But I'd struggle if an individual page of the document looked like this (modified, but close to official example):

:includedir: _includes

include::{includedir}/fragment1.adoc[]

Some text of my own

include::{includedir}/fragment2.adoc[]

Which is why I want the includes to actually work as inline includes for editing purposes too.