The parser is unable to handle big files

[Bernardo-MG]

Currently the parser is able to handle only files of a handful MBs.

It has been verified to work with files up to 50MB, and takes around 3 minutes for each 10MBs.

The reason behind this problem seems to be the Pyparsing library. But swapping it for another one would mean rewriting the factory, and the configuration DSL.

[Alexkane]

Would it be conceptually possible to break this process down and run it in parallel to improve performance?

[Bernardo-MG]

In my little experience with parallelizing, it isn't so easy to take advantage of.

Also, first of all the parser itself would need to split the file, and then decide how to handle these pieces before continuing with the parsing.