Testing procedure

[MaxGhenis]

Currently @feenberg is filing issues when TAXSIM deviates from our emulator. These issues are generally either problems with the translation or with policyengine-us calculations. This conversation suggests procedures to address these.

Generally we will want to file calculation issues in policyengine-us, and translation issues here. You can find policyengine-us state tax code here: parameters and variables. Specifically, issues filed in policyengine-taxsim should be closeable via policyengine-taxsim PRs, and issues filed in policyengine-us should be closeable via policyengine-us PRs.

Here's how I'd suggest we move forward with testing, depending on the situation:

1. Confident in a policyengine-us bug -> file issue in policyengine-us. Ideally with an integration test (example) showing the expected and actual results, and reason why (e.g., tax rates are wrong).
2. Confident in a TAXSIM-PolicyEngine translation issue -> file issue in policyengine-taxsim.
3. Confident it's not a translation issue, but unsure if the deviation is a bug in policyengine-us or TAXSIM -> file discussion in policyengine-us.
4. Unsure about source of deviation -> file discussion in policyengine-taxsim.

[feenberg]

I am usually not confident of the source of the problem, unless there is an error message from the emulator, which hasn't been happening lately. Actually, I usually have no basis to even guess. policyengine-us is so large, and I have so little understanding of Python and yaml, I can't really do much to figure it out. Perhaps I should default to the emulator, and let Al triage the messages? Dan

…

On Fri, 1 Nov 2024, Max Ghenis wrote: Currently @feenberg is filing issues when TAXSIM deviates from our emulator. These issues are generally either problems with the translation or with policyengine-us calculations. This conversation suggests procedures to address these. Generally we will want to file calculation issues in policyengine-us, and translation issues here. You can find policyengine-us state tax code here: [parameters](https://github.com/PolicyEngine/policyengine-us/tree/master/policyengine_us/parameters/gov/states) and [variables](https://github.com/PolicyEngine/policyengine-us/tree/master/policyengine_us/variables/gov/states). Specifically, issues filed in policyengine-taxsim should be closeable via policyengine-taxsim PRs, and issues filed in policyengine-us should be closeable via policyengine-us PRs. Here's how I'd suggest we move forward with testing, depending on the situation: 1. [**Confident in a policyengine-us bug -> file issue in policyengine-us.**](https://github.com/PolicyEngine/policyengine-us/issues/new) Ideally with an integration test ([example](https://github.com/PolicyEngine/policyengine-us/blob/master/policyengine_us/tests/policy/baseline/gov/states/ct/tax/income/integration.yaml)) showing the expected and actual results, and reason why (e.g., tax rates are wrong). 2. [**Confident in a TAXSIM-PolicyEngine translation issue -> file issue in policyengine-taxsim.**](https://github.com/PolicyEngine/policyengine-taxsim/issues/new) 3. [**Confident it's not a translation issue, but unsure if the deviation is a bug in policyengine-us or TAXSIM -> file discussion in policyengine-us.**](https://github.com/PolicyEngine/policyengine-us/discussions/new?category=general) 3. [**Unsure about source of deviation -> file discussion in policyengine-taxsim.**](https://github.com/PolicyEngine/policyengine-taxsim/discussions/new?category=general) -- Reply to this email directly or view it on GitHub: #63 You are receiving this because you were mentioned. Message ID: ***@***.***>

[PavelMakarchuk]

I agree, I think it would be the safest to file the issue here in the emulator repository and then we can run some local tests to see whether we should file an issue in policyengine-us or if we should keep the issue in this repository

[MaxGhenis]

For reference, @feenberg is checking that any quantity is within the greater of $25 or 0.1% of AGI.