Congestion tti by ShahrzadBorjian2 · Pull Request #1383 · CityofToronto/bdit_data-sources

ShahrzadBorjian2 · 2026-04-07T21:14:18Z

What this pull request accomplishes:

Implementing new segment level and aggregated tti using updated tti methodology

Issue(s) this solves:

What, in particular, needs to reviewed:

citywide_daily_segment_tti_dynamic.sql: calculates TTI at the segment level for the entire citywide congestion network
segment_here_vkt_2024.sql: generates segment-level VKT proxy using HERE sample size
citywide_daily_weighted_sqrt_aadt_length_tti_dynamic.sql: aggregates segment-level TTI to citywide TTI
downtown_daily_weighted_sqrt_aadt_length_tti_dynamic.sql: aggregates segment-level TTI to downtown TTI

What needs to be done by a sysadmin after this PR is merged

_create new functions to produce tti values

…ison table dynamic

ShahrzadBorjian2

Thanks @gabrielwol. Looks great. I have added my comments.

ShahrzadBorjian2 · 2026-05-04T17:59:11Z

@@ -0,0 +1,52 @@
+CREATE OR REPLACE FUNCTION here_agg.agg_overnight_tt(


Would you add some high level comments?

ShahrzadBorjian2 · 2026-05-04T17:59:51Z

+    FROM here_agg.raw_segments
+    WHERE
+        dt >= agg_overnight_tt.mnth - interval '6 months'
+        AND dt < agg_overnight_tt.mnth


I thought we decided to use force mnth to month rolling instead of day to day

ShahrzadBorjian2 · 2026-05-04T18:03:28Z

+    SELECT
+        segment_id,
+        agg_overnight_tt.mnth AS mnth,
+        AVG(tt) AS overnight_avg_tt,


I used weighted average travel time (by obs) not sure if we want to use that approach.

ShahrzadBorjian2 · 2026-05-04T18:04:55Z

+            ) OR (
+                dt >= '2024-01-01'::date
+                AND hr BETWEEN 1 AND 4
+            )


We need to re-run analysis each year to select minim 4 hr time window

That's not an easy thing to operationalize into an automated pipeline, I would suggest omitting it going forward.

ShahrzadBorjian2 · 2026-05-04T18:06:47Z

+
+    INSERT INTO here_agg.segment_overnight_tts (
+        segment_id, mnth, overnight_avg_tt, rolling_6month_quasi_obs
+    )


Should we add minimum observation threshold?

ShahrzadBorjian2 · 2026-05-05T15:22:22Z

+        cc.area_name
+    FROM congestion.congestion_segments AS seg
+    JOIN gis.community_council_2018 AS cc
+        ON cc.the_geom && seg.geom --quick bounding box intersection using gist idx


So the function may assign a segment to a community council just because their boxes overlap, even if the real geometries do not?

Good catch, I was missing an st_intersects in the where clause. Usually using a && join along with an st_intersects is faster than simply joining on st_intersects.

ShahrzadBorjian2 · 2026-05-05T15:24:16Z

+    ON st_intersects(st_transform(dt.geom, 4326), seg.geom)
+WHERE seg.ver_id = here_agg.select_map_version(
+    start_date := segment_areas.mnth,
+    end_date := segment_areas.mnth + 1,


You meant one day interval not one month, right?

1 day is OK, because map version should be the same for entire month.
adding that as a comment.

ShahrzadBorjian2 · 2026-05-05T15:25:28Z

+    'Downtown' AS area_name
+FROM congestion.congestion_segments AS seg
+JOIN gis.to_core_downtown AS dt
+    ON st_intersects(st_transform(dt.geom, 4326), seg.geom)


So we want any intersect with "downtown" to be considered as "downtown" ?

Options:
st_intersects(st_transform(dt.geom, 4326), seg.geom) (950 segments)

st_within(seg.geom, st_transform(dt.geom, 4326)) (861 segments)

I think this is the winner!

ON st_intersects(st_transform(dt.geom, 4326), seg.geom) AND ST_Length(ST_Intersection(seg.geom, st_transform(dt.geom, 4326))) / ST_Length(seg.geom) > 0.50 (903 segments)

ShahrzadBorjian2 · 2026-05-05T15:28:16Z

@@ -0,0 +1,22 @@
+-- Table: here_agg.monthly_link_vkt


same as above, the naming might be confusing as thereis no vkt yet

Renaming to monthly_link_sample_size 👍

ShahrzadBorjian2 · 2026-05-05T15:42:35Z

+| hr            | smallint         | 0                  |            |
+| tti           | double precision | 1.0812660631815794 |            |
+| num_segments  | integer          | 4777               |            |
+| road_category | text             | Non-Highway        |            |


A few minor comments:

Clarify that the 6-month rolling is based on monthly aggregates, not daily data

Note that monthly_link_vkt stores sample_size (not true VKT), VKT is calculated later as sample_size * length

Add a simple pipeline/data flow diagram to connect tables

Clarify that overnight TT = free-flow baseline (used for TTI denominator)

Simplify explanation of num_obs is “approx number of vehicles from probe data”

Check segment_id type consistency (integer vs bigint)

In area_tti, clarify that TTI is VKT-weighted hourly TTI

mention that 6-month rolling aligns with overnight baseline methodology

…solve map version issues

ShahrzadBorjian2 added 4 commits April 7, 2026 20:56

adding sql to calculate segment level tti- congestion analysis

d269426

adding sql to calculate segment level vkt- congestion analysis

dc5a2f1

adding sql to calculate citywide tti- congestion analysis

f543ccc

adding sql to calculate downtown wide tti- congestion analysis

561a721

ShahrzadBorjian2 requested a review from gabrielwol April 7, 2026 21:14

ShahrzadBorjian2 self-assigned this Apr 7, 2026

gabrielwol added 15 commits April 8, 2026 21:45

#1383 tti agg functions and dag

b135545

#1383 move files to folders

554d3b2

#1383 area tti

6121379

#1383 transform monthly_segment_vkt_agg into function to make map ver…

3fc22b5

…ison table dynamic

#1383 use generate series to ensure no missing days

c9d2d88

#1383 add highway: all category using rollup

fb3a9d2

#1383 remove unnecessary grouping col

29e7c5d

#1383 switch to monthly rolling overnight speeds, previous month vkt

c750531

#1383 remove is_wkdy from overnight, tti tables

e49d084

#1383 use 6 months lookback for vkt

4a66761

#1383 remove unused files

1ba4c73

#1383 minor table adjustments

c2bef5a

#1383 tweaks to generate tti daily

0618cd2

#1383 sqlfluff

721303f

#1383 a rough sql readme

25f2599

gabrielwol assigned gabrielwol and unassigned ShahrzadBorjian2 Apr 29, 2026

gabrielwol removed their request for review April 29, 2026 21:06

gabrielwol added 5 commits April 30, 2026 15:52

#1383 mv segment areas view to function

2453bb1

#1383 modify segment_areas to take ver_id

b408ed8

#1383 use new partitioned tables

60dcfb0

#1383 simplify plpgsql functions to sql

8f90bb5

#1383 add surgical methods to tti functions, backfill procedure

cbd2241

ShahrzadBorjian2 commented May 5, 2026

View reviewed changes

gabrielwol added 9 commits May 5, 2026 15:53

#1383 address missing st_intersects in segment areas

e99a2c4

#1383 more accurately rename vkt to sample size

1bb60a6

#1383 use retired segments in calculating overnight rolling tt

c15a9cf

#1383 add ver_id to more tables

169bea8

#1383 switch from link vkt -> segment vkt to straight segment vkt to …

1905cb3

…solve map version issues

#1383 rename files, more vkt fix

d9f357d

#1383 change downtown tti to >= 50% overlap

d57131f

#1383 add check constraints on mnth = month

0547c46

#1383 combine vkt and overnight lookbacks into one

553e3f0

		@@ -0,0 +1,52 @@
		CREATE OR REPLACE FUNCTION here_agg.agg_overnight_tt(

Conversation

ShahrzadBorjian2 commented Apr 7, 2026

What this pull request accomplishes:

Issue(s) this solves:

What, in particular, needs to reviewed:

What needs to be done by a sysadmin after this PR is merged

Uh oh!

ShahrzadBorjian2 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gabrielwol May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gabrielwol May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gabrielwol May 7, 2026 •

edited

Loading

gabrielwol May 7, 2026 •

edited

Loading