diff --git a/NEWS.md b/NEWS.md new file mode 100644 index 0000000..5734a69 --- /dev/null +++ b/NEWS.md @@ -0,0 +1,223 @@ +# ptaxsim 1.1.0 + +## Background + +This release adds support for tax year 2024. It is intended for use with the +`2024.0.x` database version. + +The release also features a significant number of changes to the database schema +and to package functions to accommodate changes that the Clerk and the +Treasurer made to the data they report in 2024. Source data from the Clerk and +the Treasurer forms the core of the PTAXSIM database, so any changes they make +need to be reflected in this package, too. + +The Clerk and the Treasurer changed their data reporting this past year because +they both completed a migration from a legacy AS/400 mainframe system to a +new integrated property tax system of record. + +The main changes that the Clerk and the Treasurer made in 2024 include: + +- **Changed the TIF revenue share calculation so that it is performed at the + PIN-level rather than the tax-code-level**. This means that TIF increment is + now calculated based on the difference between the frozen and current taxable + EAV _for each individual PIN in a TIF_, rather than the difference between the + frozen and current taxable EAV of its tax code as a whole. As a result of this + change, a TIF can now have a positive increment even if the total taxable EAV + of its tax code has declined since its establishment, as long as individual + PINs within the tax code have experienced EAV growth during that time. +- **Changed a small number of agencies to funds**. An example is the City of + Chicago library fund, whose agency number (`030210001`) had a non-zero levy + in the `agency` table in 2023, but has a $0 levy in 2024. Instead, the library + fund's levy is listed in the `agency_fund` table for 2024 using the agency + number for the City of Chicago (`030210000`). By reporting this kind of levy + as a fund underneath a parent agency rather than an independent agency, the + format of the Clerk's data is now more similar to the format that + agencies like the City of Chicago use for their own internal accounting. + - In 2024, only 78 agencies were affected by this change, representing + roughly 4% of all agencies. All of the changed agencies fit into one + of the following categories: + - Mental health districts + - Public health funds + - General assistance funds + - Road and bridge funds + - Library funds + - To see the full list of agencies that changed in this manner in 2024, + run the following SQL query against the 2024 PTAXSIM database: + +```sql +SELECT * FROM agency_info WHERE agency_change_24 +``` + +- **Switched from three-digit to six-digit fund numbers to add a greater level + of detail to funds**. Prior to 2024, fund numbers (`agency_fund.fund_num`) + consisted of three digits, and funds with the same `fund_num` in different + agencies always had the same `fund_name`. In 2024, the Clerk changed their + fund numbers so that they consist of six digits, and they are no longer + guaranteed to share the same name across agencies. The Clerk also added many + new six-digit funds that were previously aggregated into larger three-digit + funds. For these new six-digit funds, the first three digits of the fund + number always matches the three-digit number for the aggregate fund that + existed prior to 2024. + - To see the full list of funds that are new in 2024, run the following + query against the 2024 PTAXSIM database: + +```sql +SELECT * FROM agency_fund_info WHERE fund_num NOT LIKE '%000' +``` + +- **Removed a number of minor columns from agency reports**. None of these + columns were used in any core PTAXSIM functions, so we expect that their + absence will not affect most users. +- **Added a new "authority number" for each agency in agency reports**. So far, + authority numbers appear to have a 1:1 relationship to agency numbers, though + we expect this may change in future years. + +Read on for a detailed description of the changes we made to the PTAXSIM +database and functions to handle these changes in the source data. + +## Breaking changes + +- **Added a new table `pin_tif_distribution` and associated lookup function + [`lookup_pin_tif()`](https://ccao-data.github.io/ptaxsim/reference/lookup_pin_tif.html) + to handle the methodological change to the TIF revenue share calculation**. + The [`tax_bill()`](https://ccao-data.github.io/ptaxsim/reference/tax_bill.html) + function now also accepts an optional `pin_tif_dt` argument that you can use for post-2024 TIF + counterfactuals. + - Note that there are now two different TIF lookup functions, each of + which applies to a different set of years: + - [`lookup_pin_tif()`](https://ccao-data.github.io/ptaxsim/reference/lookup_pin_tif.html) + will only return rows where `year >= 2024`. + - [`lookup_tif()`](https://ccao-data.github.io/ptaxsim/reference/lookup_tif.html) + will only return rows where `year < 2024`. + - **How this change affects you**: If you maintain any TIF counterfactuals, + you will need to update your use of the `tax_bill()` function to pass in + an altered `pin_tif_dt` for any tax years that you analyze beyond 2023. + For an example of this type of change, see the [Tinkering with TIFs + vignette](https://ccao-data.github.io/ptaxsim/articles/tifs.html), which + we have updated to include a TIF counterfactual with data for tax year + 2024. +- **Added new `agency_info.agency_*_24` columns to handle agencies that have + changed to funds in 2024**. You can use these columns to construct a crosswalk + to analyze agencies over time, even if they changed to a fund in 2024. + - The new columns include: + - `agency_info.agency_change_24` (boolean, required): Whether the agency's + number changed in 2024, due to becoming a fund. + - `agency_info.agency_num_24` (string, optional): The agency's new number + starting in 2024. Null if the agency number did not change in 2024. + - `agency_info.agency_name_24` (string, optional): The agency's name + starting in 2024. Null if the agency number did not change in 2024. + - **How this change affects you**: If you maintain code that analyzes + agencies over time, and you want to update your code to include 2024 data, + you should use the `agency_info.agency_change_24` column to determine + whether the Clerk changed any of the agencies you analyze to funds in + 2024. If any of your agencies have changed to funds, you will need to use + the `agency_num_24` column to join pre- and post-2024 data. See [this + vignette](https://ccao-data.github.io/ptaxsim/articles/agencies.html) + for an example using the City of Chicago Library Fund to show how to + handle this type of change. +- **Added a new column `agency_fund.fund_type_num` to handle changing fund + numbers in 2024**. In 2024, the Clerk changed their fund numbers so that + they consist of six digits instead of three, and they are no longer + guaranteed to refer to the same `fund_name` across agencies. However, the + first three digits of a fund number are still guaranteed to refer to a + consistent fund "type", so we introduced a new column + `agency_fund.fund_type_num` to represent the notion of a "fund type" that + is consistent across agencies. + - **How this change affects you**: If you use `agency_fund.fund_num` to + track specific funds across agencies, you will need to switch to + using `agency_fund.fund_type_num` for that purpose. +- **Appended three trailing zeros (`000`) to all fund numbers in `agency_fund` + and `agency_fund_info` for years prior to 2024**. This change is necessary + in order to align three-digit pre-2024 fund numbers with their new six-digit + representations in 2024. + - **How this change affects you**: If you maintain code that references + specific three-digit `fund_num` values, you will need to update these + values to append three trailing zeros. Alternatively, you could update + your code to reference `fund_type_num` instead of `fund_num`, per the + bullet point above. +- **Changed the `agency_fund_info` table so that it is now unique by + `(agency_num, fund_num)` instead of `(fund_num)`**. Since fund numbers are + no longer consistent across agencies, we can't use them as the exclusive basis + for the `agency_fund_info` table anymore. To handle this change, we have + altered the `agency_fund_info` table so that it includes `agency_num` + in its primary key. + - **How this change affects you**: If you use the `agency_fund_info` table + and treat it as unique by `fund_num`, you will need to update your + code to add `agency_num` when joining to this table. +- **Dropped a few columns that the Clerk has removed from its agency reports**. + - The dropped columns are: + - `agency_fund.ptell_reduced_ind` (PTELL-reduced levy indicator) + - `agency.reduction_type` (PTELL-reduced levy type) + - `agency.total_reduced_levy` (PTELL reduced levy total) + - **How this change affects you**: If you use any of the columns listed + above, you will need to remove them from your code. Feel free to [open an + issue](https://github.com/ccao-data/ptaxsim/issues/new) to ask for help + replacing these columns; we're interested to know if and how PTAXSIM users + are using them. +- **Consolidated non-Cook County EAV columns into a new column + `agency.cty_overlap_eav`**. + - These columns have been removed and their values are now included in the + `agency.cty_overlap_eav` column: + - `agency.cty_dupage_eav` + - `agency.cty_lake_eav` + - `agency.cty_will_eav` + - `agency.cty_kane_eav` + - `agency.cty_mchenry_eav` + - `agency.cty_dekalb_eav` + - `agency.cty_grundy_eav` + - `agency.cty_kankakee_eav` + - `agency.cty_kendall_eav` + - `agency.cty_lasalle_eav` + - `agency.cty_livingston_eav` + - **How this change affects you**: If you use any of the columns listed + above, you will need to remove them from your code. If `cty_overlap_eav` + does not work for your purposes, [open an + issue](https://github.com/ccao-data/ptaxsim/issues/new) and we'll do our + best to help you update your analysis. + +## Improvements + +- **Added a new exemption column `pin.exe_vet_dis_100`**. This column + corresponds to the new 100% disability level for the [Veterans with + Disabilities Exemption](https://www.cookcountyassessoril.gov/veterans-disabilities-exemption). + The lookup function + [`lookup_pin()`](https://ccao-data.github.io/ptaxsim/reference/lookup_pin.html) + now returns this column. + - **How this change affects you**: You may use this column if you would + like to analyze this new exemption. +- **Added a new column `agency.authority_num`**. We expect this column to be + more useful in future years when the Clerk continues migrating from the + core "Agency" entity to the "Authority" entity. + - **How this change affects you**: We recommend ignoring this column for + now, because it is currently duplicative of `agency_num` and we don't + have full confidence in our understanding of the Clerk's plans for this + column in the future. However, it remains available if you are interested + in investigating it. +- **Added support for pre-release database versions**. The code now supports + database versions with pre-release suffixes like `2024.0.0-alpha.1`. + Databases tagged with pre-release versions are unreleased but available for + testing. + - **How this change affects you**: This change will not affect most PTAXSIM + users. It is only useful for PTAXSIM maintainers and pre-release testers. +- **Updated the [Tinkering with TIFs + vignette](https://ccao-data.github.io/ptaxsim/articles/tifs.html) to + demonstrate the correct way to handle the new TIF revenue share calcluation** + ([#77](https://github.com/ccao-data/ptaxsim/pull/77)). + - **How this change affects you**: You should read the latest version of the + vignette if you use PTAXSIM for TIF counterfactuals. +- **Added [a new + vignette](https://ccao-data.github.io/ptaxsim/articles/agencies.html) + to demonstrate the correct way to analyze agencies and funds over time given + the 2024 change that switched some agencies to funds**. + ([#84](https://github.com/ccao-data/ptaxsim/pull/84)). + - **How this change affects you**: You should read the vignette if you use + PTAXSIM to analyze specific agencies over time so that you understand how + to update your code. + +## Bug fixes + +- **Lowered the memory usage of the `pin_geometry` extraction script** + ([#67](https://github.com/ccao-data/ptaxsim/pull/67)). This script previously + struggled to complete on our 160gb server. It now executes without any issues. + - **How this change affects you**: This change will not affect you as a + PTAXSIM user. It is only useful for PTAXSIM maintainers.