Skip to content

[Feature] Investigate speeding up stata-acro #351

@jim-smith

Description

@jim-smith

Is your feature request related to a problem? Please describe.
The acro versions of queries in Stata are much slower to run than the native ones

Describe the solution you'd like
Stata is fast because (1) it is compiled (not interpreted) and (2) it keeps everything in memory.

We could try and reduce the overhead by not passing as much data through the SDI interface to python.
For example, if there is an exclusion ( if() statement) in the Stata query,
we should see if we can apply the exclusion before the data is bundled up by the sfi interface and passed to python (I.e. in acro.ado) rather than afterwards in acro_stata_parser.py

Describe alternatives you've considered

  1. Find a big open source dataset.
  2. Run a profiler on the code to see where time is spent - could be something else inefficient in acro_stata_parser.py

Additional context

Metadata

Metadata

Labels

enhancementNew feature or request

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions