Documentation in progress.

Author: ColinH

README.md

@@ -56,7 +56,8 @@ The included [grammar analysis](doc/Grammar-Analysis.md) finds several typical e
 
 ## Design
 
-The PEGTL is designed to be "lean and mean", the core library consists of around 10K lines of code.
+The PEGTL is designed to be "lean and mean".
+The core library consists of around 10K lines of code.
 Emphasis is on simplicity and efficiency, putting a well-tuned *relatively* simple approach above overly complex optimisations.
 
 The PEGTL is mostly concerned with parsing combinators and grammar rules, and with giving the user of the library (the possibility of) full control over all other aspects of a parsing run.

doc/Action-Reference.md

@@ -19,23 +19,13 @@ This default can be changed via the macro `TAO_PEGTL_NAMESPACE` in `tao/pegtl/co
 
 ## Contents
 
-* [Nothing](#nothing)
-* [Maybe Nothing](#maybe-nothing)
-* [Add Guard](#add-guard)
-* [Add State](#add-state)
-* [Change Action](#change-action)
-* [Change Action and State](#change-action-and-state)
-* [Change Action and States](#change-action-and-states)
-* [Change Control](#change-control)
-* [Change State](#change-state)
-* [Change States](#change-states)
-* [Control Action](#control-action)
-* [Disable Action](#disable-action)
-* [Enable Action](#enable-action)
-* [Require Apply](#require-apply)
-* [Require Apply0](#require-apply0)
+* [Default](#default)
+* [Tag Classes](#tag-classes)
+* [Index](#index)
 
-### Nothing
+## Default
+
+###### `nothing< R >`
 
 An action class template that does nothing, simply a *nop* action.
 
@@ -46,7 +36,7 @@ template< typename Rule >
 struct nothing {};
 ```
 
-Usually also serves as base class for the default case of custom action class templates to indicate that the default is "no action".
+Also serves as base class for the default case of custom action class templates.
 
 ```c++
 template< typename Rule >
@@ -57,17 +47,48 @@ struct my_action
 // that define static apply() or apply0() functions.
 ```
 
-### Maybe Nothing
+When `my_action< Rule >` is publicly derived from `tao::pegtl::nothing< Rule >` it indicates to the PEGTL that no `apply()` or `apply0()` should be expected in `my_action< Rule >`.
+
+## Tag Classes
+
+Other tag classes beyond `nothing< Rule >` to inform the PEGTL of what is intended for or expected of an action.
+
+####### `maybe_nothing`
 
 An action class alias defined as `nothing< void >`.
-An action class for `Rule` that is not derived from `nothing< Rule >` but intentionally has no `apply()` or `apply0()` must/should (TODO) derive from `maybe_nohting` to signal that the absence of these functions is not an error.
+An action class for `Rule` that is not derived from `nothing< Rule >` but intentionally has no `apply()` or `apply0()` must derive from `maybe_nothing` to signal that the absence of these functions is not an error.
+
+This is usually the case for actions that define a `match()` function, and which might or might not also have an `apply()` or `apply0()` added later down the inheritance chain.
+
+###### `require_apply`
+
+The class `require_apply` is an empty class used as tag.
+An action class that is publicly derived from `require_apply` must define a static `apply()` function that can be called as action function.
+
+Wins against [`maybe_nothing`](#maybe-nothing) whan a class has both as public base classes.
+Included via `tao/pegtl/action/require_apply.hpp`.
+
+###### `require_apply0`
+
+The class `require_apply0` is an empty class used as tag.
+An action class that is publicly derived from `require_apply0` must define a static `apply0()` function that can be called as action function.
+
+Wins against [`maybe_nothing`](#maybe-nothing) whan a class has both as public base classes.
+Included via `tao/pegtl/action/require_apply0.hpp`.
+
 
 ### Add Guard
 add_guard
 
+Publicly derives from [`maybe_nothing`](#maybe-nothing).
+Included via `tao/pegtl/action/add_guard.hpp`.
+
 ### Add State
 add_state
 
+Publicly derives from [`maybe_nothing`](#maybe-nothing).
+Included via `tao/pegtl/action/add_state.hpp`.
+
 ### Change Action
 
 An action class template with a `match()` function that parses the rule to which it is attached with its template parameter as action class template.
@@ -115,21 +136,23 @@ In other words, an action version of the [`enable`](Rule-Reference.md#enable) ru
 Publicly derives from [`maybe_nothing`](#maybe-nothing).
 Included via `tao/pegtl/action/enable_action.hpp`.
 
-### Require Apply
-
-The class `require_apply` is an empty class used as tag.
-An action class that is publicly derived from `require_apply` must define a static `apply()` function that can be called as action function.
-
-Wins against [`maybe_nothing`](#maybe-nothing) whan a class has both as public base classes.
-Included via `tao/pegtl/action/require_apply.hpp`.
-
-### Require Apply0
-
-The class `require_apply0` is an empty class used as tag.
-An action class that is publicly derived from `require_apply` must define a static `apply0()` function that can be called as action function.
+## Index
 
-Wins against [`maybe_nothing`](#maybe-nothing) whan a class has both as public base classes.
-Included via `tao/pegtl/action/require_apply0.hpp`.
+* [`nothing`](#nothing) <sup>[(default)](#default)</sup>
+* [`maybe_nothing`](#maybe-nothing) <sup>[(tag classes)](#tag-classes)</sup>
+* [`require_apply`](#require-apply) <sup>[(tag classes)](#tag-classes)</sup>
+* [`require_apply0`](#require-apply0) <sup>[(tag classes)](#tag-classes)</sup>
+* [Add Guard](#add-guard)
+* [Add State](#add-state)
+* [Change Action](#change-action)
+* [Change Action and State](#change-action-and-state)
+* [Change Action and States](#change-action-and-states)
+* [Change Control](#change-control)
+* [Change State](#change-state)
+* [Change States](#change-states)
+* [Control Action](#control-action)
+* [Disable Action](#disable-action)
+* [Enable Action](#enable-action)
 
 ---
 

doc/Changelog.md

@@ -4,7 +4,7 @@
 
 **Not yet released**
 
-* *Moar templates*, more templates in more places!
+* Even more templates, in even more places, templates everywhere!
 * Use the [**migration guide**](Migration-Guide.md#version-400) when updating.
 * Infrastructure
   * Switched to Boost Software License, Version 1.0.
@@ -92,6 +92,7 @@
 * Added new customization point for error messages.  -- TODO!
 * Added optional source line output for the tracer.  -- TODO?
 * Other
+  * Routed rewind-guard creation through the Control class.
   * Renamed contrib "limit_depth" functionality to "check_depth".
   * Renamed contrib "check_bytes" functionality to "check_count".
   * Renamed contrib "limit_bytes" functionality to "limit_count".
@@ -101,6 +102,7 @@
   * Renamed `end_of_line()` input member function to `end_of_line_or_file()`.
   * Renamed contrib "to_string" functionality to "type_to_string".
   * Added `type_to_string_view` function that mirrors `type_to_string`.
+  * Renamed `alphabet.hpp` to `alphabet_constants.hpp`.
 * Cleanup
   * Removed rule `forty_two`, we apologize for any inconvenience.
   * Removed rule `bytes` and replaced with `many` for different data types.

doc/Contrib-and-Examples.md

@@ -26,10 +26,16 @@ For all questions and remarks contact us at **taocpp(at)icemx.net**.
 
 ###### `<tao/pegtl/contrib/alphabet.hpp>`
 
-* Constants for ASCII letters.
-* Shortens `string<'f','o','o'>` to `string<f,o,o>`.
-* Ready for production use.
-* Superceeded by `TAO_PEGTL_STRING()`.
+* Character constants for ASCII letters.
+* In sub-namespace `alphabet`.
+* Shorten e.g. `string<'f','o','o'>` to `string<f,o,o>`.
+* Probably better to use `TAO_PEGTL_STRING()`.
+
+###### `<tao/pegtl/contrib/alphabet_rules.hpp>`
+
+* Rules for single ASCII letters.
+* In sub-namespace `alphabet::rules`.
+* Shorten e.g. `one<'a'>` to `a`.
 
 ###### `<tao/pegtl/contrib/http.hpp>`
 

doc/Development.md

@@ -1,6 +1,7 @@
 # Development
 
-Notes on future development and public parts of the development roadmap.
+Notes on past, present and future development.
+Public parts of the development roadmap.
 
 
 ## C++ Standard
@@ -15,7 +16,7 @@ Version 3.x of the PEGTL requires at least C++17.
 
 Version 4.x of the PEGTL requires at least C++17.
 
-Version 5.x of the PEGTL will make the jump to C++20 or newer.
+Version 5.x of the PEGTL will jump to C++20 or C++23.
 
 
 ## C++ Features
@@ -30,6 +31,7 @@ Version 5.x of the PEGTL will make the jump to C++20 or newer.
  * Keep an open eye for opportunities to use C++20 `[[likely]]` and `[[unlikely]]`.
  * Keep an open eye for opportunities to use C++20 `constinit` and `consteval`, and
  * keep an open eye for opportunities to use the extended `constexpr` facilities.
+ * For example make the bulk() and size() functions in the peek classes consteval!
  * Keep an open eye for opportunities to use the extended CTAD facilities from C++20.
  * Keep an open eye for opportunities to use class types as non-type template parameters.
  * Replace the hand-crafted endian facilities with C++20 `std::endian` and C++23 `std::byteswap`.
@@ -56,6 +58,17 @@ A couple of things that *could* be done in the area of buffer inputs.
  * Investigate the use of ("stackful") coroutines for parsing from a network socket, and
  * investigate whether this can also be used for incremental parsing that keeps everything.
 
+
+## Development Tools
+
+We use the code coverage facilities of the GCC and Clang compilers to meet our goal of 100% code coverage with unit tests.
+Unfortunately the coverage reports are not 100% reliable.
+
+We also try to use `clang-format` to ensure consistent formatting of our source code.
+Unfortunately it, too, can get confused and result in incorrectly formatted code.
+
+For example under some circumstances `clang-format` does not understand whether a `&&` is the logical operator or an r-value reference declarator, or whether a `<` is a comparison operator or introduces a template argument list.
+
 ---
 
 This document is part of the [PEGTL](https://github.com/taocpp/PEGTL).

doc/Getting-Started.md

@@ -1,7 +1,6 @@
 # Getting Started
 
-Since the PEGTL is a parser library, here is an "inverse hello world" example that parses,
-rather than prints, the string `Hello, foo!` for any sequence of alphabetic ASCII characters `foo`.
+Since the PEGTL is a parser library, here is an "inverse hello world" example that parses, rather than prints, the string `Hello, foo!` for any sequence of alphabetic ASCII characters `foo`.
 
 ```c++
 #include <string>
@@ -50,8 +49,8 @@ namespace hello
    template<>
    struct action< name >
    {
-      template< typename ParseInput >
-      static void apply( const ParseInput& in, std::string& v )
+      template< typename ActionInput >
+      static void apply( const ActionInput& in, std::string& v )
       {
          v = in.string();
       }
@@ -61,21 +60,22 @@ namespace hello
 
 int main( int argc, char* argv[] )
 {
-   if( argc != 2 ) return 1;
-
+   if( argc < 2 ) {
+      return 1;
+   }
    // Start a parsing run of argv[1] with the string
    // variable 'name' as additional argument that will
    // be passed to all called actions, including the
    // one we attached to the 'name' rule above.
 
    std::string name;
    pegtl::argv_input in( argv, 1 );
-   if( pegtl::parse< hello::grammar, hello::action >( in, name ) ) {
-      std::cout << "Good bye, " << name << "!" << std::endl;
-   }
-   else {
-      std::cout << "I don't understand." << std::endl;
+
+   if( !pegtl::parse< hello::grammar, hello::action >( in, name ) ) {
+      std::cout << "I can't parse you!" << std::endl;
+      return 1;
    }
+   std::cout << "Good bye, " << name << "!" << std::endl;
    return 0;
 }
 ```
@@ -126,7 +126,7 @@ Beyond these standard combinators the PEGTL contains a [large number of addition
 
 The PEGTL also contains a [large number of atomic rules](Rule-Reference.md) for matching ASCII and Unicode characters, strings, ranges of characters, integers, beginning-of-file or end-of-line, ...
 
-Many of these rules can directly be applied to objects in the input, frequently of type `char`, but also to data members or the return values of global or member functions in cases where the input is a sequence of non-trivial objects.
+Many of these rules can directly be applied to objects in the input, frequently of type `char`, but also to data members or the return values of global or member functions in cases where the input is a sequence of other types.
 
 ## Grammar Analysis
 
@@ -141,7 +141,8 @@ It is best practice to create a separate dedicated program that does nothing els
 #include <tao/pegtl.hpp>
 #include <tao/pegtl/analyze.hpp>
 
-// This example uses the included JSON grammar
+// For this example we use the included JSON grammar.
+
 #include <tao/pegtl/contrib/json.hpp>
 
 namespace pegtl = tao::pegtl;
@@ -151,10 +152,9 @@ using grammar = pegtl::must< pegtl::json::text, pegtl::eof >;
 int main()
 {
    if( pegtl::analyze< grammar >() != 0 ) {
-      std::cerr << "Cycles without progress detected!\n";
+      std::cerr << "Cycles without progress detected!" << std::endl;
       return 1;
    }
-
    return 0;
 }
 ```