serve-maps-as-tiles #38
Conversation
update readme
- update seed - update model
endpoint returning welsh and english values
remove postgres reintegrate sqllite3 add scotland add northern ireland update postcode search
fix index of multiple deprivation endpoint
add pytest
add postcode validation
add filters to views update readme
… view as a table. update maplibre to black and white base map, toggles for nations and years.
ERROR: (ContainerAppInvalidResourceTotal) The total requested CPU and memory resources for this application (CPU: 1.50, memory: 4.0) is invalid. Total CPU and memory for all containers defined in a Consumption Container App must add up to one of the following CPU - Memory combinations: [cpu: 0.25, memory: 0.5Gi]; [cpu: 0.5, memory: 1.0Gi]; [cpu: 0.75, memory: 1.5Gi]; [cpu: 1.0, memory: 2.0Gi]; [cpu: 1.25, memory: 2.5Gi]; [cpu: 1.5, memory: 3.0Gi]; [cpu: 1.75, memory: 3.5Gi]; [cpu: 2.0, memory: 4.0Gi]; [cpu: 2.25, memory: 4.5Gi]; [cpu: 2.5, memory: 5.0Gi]; [cpu: 2.75, memory: 5.5Gi]; [cpu: 3, memory: 6.0Gi]; [cpu: 3.25, memory: 6.5Gi]; [cpu: 3.5, memory: 7Gi]; [cpu: 3.75, memory: 7.5Gi]; [cpu: 4, memory: 8Gi]
mbarton
left a comment
mbarton
left a comment
There was a problem hiding this comment.
I've got the PR check deploying and have added some comments based on my experience setting up other projects. Feel free to ignore or push any of them back to further PRs.
To get it working I:
- Created an Azure App Registration for rcpch-census-platform
- It may be possible to do this using a user assigned managed identity? All this stuff melts my brain every time I try and think about it so I just set it up the same way as the other apps we have in ACA.
- Updated the secrets in this repo with the client ID etc of that registration
- Granted it
AcrPushon the container repository - Made it a
Contributoron the container app and the container app environment- That is wider scope than I thought it needed but it got upset trying to update the secrets without it (see this run)
- The only other thing in this environment is postcodes.io so I don't think it's the biggest deal. We can recreate with a new environment if you prefer.
- I don't really want to grant the app registration full Contributor acesss to the resource group as there's more stuff in there so did it directly on the resources themselves.
- Tweak the number of CPUs to match one of the available options
| - API: `http://localhost:8001/rcpch-census-platform/api/v1/` | ||
| - Tileserver (pg_tileserv): `http://localhost:7800/` | ||
|
|
||
| Note: the nginx reverse-proxy container is used in the Azure Container Apps deployment to route `/tiles/*` and the API under a single public ingress. For local development you can usually hit Django (`:8001`) and pg_tileserv (`:7800`) directly. |
There was a problem hiding this comment.
Azure Container Apps finally supports path based routing now: microsoft/azure-container-apps#591 (comment).
| --health-retries 5 | ||
| # Define global env for all steps | ||
| env: |
There was a problem hiding this comment.
In other projects I've made this step a simple copy of the env template in the hope it helps us remember to keep the template up to date
|
banging my head here against azure pgdata issues. PGDATA cannot be mounted in azure because it cannot use CHMOD (not linux). (see Or come to think of it, we could have no volume (though seeding now takes quite a while), or we could do what postcodes.io do and ship the database. |
|
I'm happy with either no volume or burning in the entire db into the image a la postcodes.io. Also fine to just used an Azure managed postgres if simpler. |
…ation of a release. The shapes deploy workflow has been updated as per @mbarton review and refactored to work on merge to live. The infra yaml similarly removes references to nginx and pgdata. step one is to deploy to shapes branch and create a release. if this builds and deploys to ghcr the next step is to change the deploy to azure action to work on shapes branch. if this works step 3 is to change back to trigger on live. so many steps where things can break....
Overview
This PR moves the census-platform from just calculating IMD and serving lists of LSOAs/LADs etc, to serving geometry files.
ONS supports high res and low res boundary geometry (shape files) of LSOAs, SOAs, Data Zones, either clipped to low tide, or including the peripheral islands. The high definition files are over 1GB so cannot be stored on github, and would take ages to load and serve.
The secret after much research is to serve the data from CrunchyData's
pg_tileservwhich is hosted as a separate container. This generates tiles which serve quickly and work together with a CDN which caches the tiles so they can be served first with minimal latency. And of course this uses geography so we will need to upgrade to postgis.This PR therefore:
seedfunction to pull in all the full detail (BFC) geometry data from arcgis in chunks, then optimize it for serving and link it to the IMD models (Ireland is not available on arcgis so we have towgetthe geojson and simplify the oas into SOAs)pg_tileservGIthub Pages
I am sorry about the size of the PR. I initially committed a number of large files which i failed to remove from the previous commits and in error wound the whole branch back to the initial commit.