Skip to content

Add fuzz tester to catch cases where parser fails#83

Open
dillonkearns wants to merge 4 commits intomasterfrom
recoverable-parser
Open

Add fuzz tester to catch cases where parser fails#83
dillonkearns wants to merge 4 commits intomasterfrom
recoverable-parser

Conversation

@dillonkearns
Copy link
Copy Markdown
Owner

No description provided.

@dillonkearns
Copy link
Copy Markdown
Owner Author

High-level goal: it would be awesome if we changed from

parse : String -> Result (List (DeadEnd String Problem)) (List Block)

to

parse : String -> List Block

  1. In order to do that, we should have confidence that we will actually always recover from any parser errors with a fallback. This would be a great case for a fuzz tester.

At it's simplest, we could just use a random string and check that it should always parse without any errors. This PR has a test that does exactly that.

We should use this test to catch all current problems where the parser can have an error, and once they're all fixed, we can change the type as in (1).

@dillonkearns
Copy link
Copy Markdown
Owner Author

If we want to get a little more sophisticated here, we can change the fuzzer (or add another test) that uses some specific things that could be problematic to make sure they don't cause parser errors.

For example, we could throw in lots of random characters that are used in HTML tags, since that is a current area where the parser can fail.

We could also add some test cases with specific invalid HTML input and check the fallback output to make sure it's what we want, since we not only want the parser to be resilient but to have reasonable fallbacks in these specific cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant