Added support to cache online tile maps, also added custom http heade… #41

batuakan · 2025-10-20T14:52:20Z

Added support to cache online tile maps,
Added custom http headers to pass to the remote tile server to pass around CORS limitations

src/index.ts

tkurki · 2025-10-20T17:50:47Z

src/index.ts

+    })
+    if (!response.ok) {
+      console.error(`Error fetching tile ${provider.name}/${z}/${x}/${y}:`)
+      res.sendStatus(500)


Should we pass on whatever status code we got from upstream?

src/index.ts

tkurki

This mostly worked, except when I entered a non working chart source the result was this:

This feature is a bit dangerous, as it can fill up your disk and effectively lock people out of their system if the system won't start properly with disk full.

I propose at least the following safeguards:

minimum free disk space - so that seeding stops if there is not enough free space
some threshold for seeding region size, maybe with just an alert to confirm that the region is large

This kind of long running operations easily error out for one reason or another, and you don't want to start over from the very beginning. The jobs are not persisted over restarts, so it would be useful if the UI could present a list of cached providers and their seeding statuses: what regions are seeded and their status, for example in terms of number of files on disk and a [Continue seeding] button, that would create the tiles list, check for their existence and fetch only missing tiles.

src/chartDownloader.ts

README.md

tkurki · 2025-10-27T17:56:39Z

public/index.html

+        <label for="chart">Chart:</label>
+        <select id="chart" name="chart" required>
+            <option value="">-- Select chart --</option>
+            <option value="eniro">Eniro</option>


Are these defaults used in some cases?

Will remove them

tkurki · 2025-10-27T18:35:03Z

public/index.html

+            <option value="24">24</option>
+        </select>
+        <button id="seedChartsButton">Seed Chart</button>
+        <button id="clearCacheButton">Clear Cache</button>


I guess this is all right like this to get started, but I think it would make sense to have a separate list of cached chart providers with associated [Clear Cache] functionality and a separate panel/control for starting seeding jobs.

Some how tried to fix it

src/chartDownloader.ts

tkurki · 2025-10-27T18:45:48Z

src/chartDownloader.ts

+            const bbox = this.getBBox(feature.geometry as Polygon);
+            for (let z = zoomMin; z <= zoomMax; z++) {
+                const [minX, minY] = this.lonLatToTileXY(bbox[0], bbox[3], z); // top-left
+                const [maxX, maxY] = this.lonLatToTileXY(bbox[2], bbox[1], z); // bottom-right


Will this work over antimeridian?

Fixed, I hope

tkurki · 2025-10-27T18:46:36Z

src/chartDownloader.ts

+
+    async getRegion(regionGUID: string): Promise<Record<string, any>> {
+        let regionData: any
+        const resp = await fetch(`${this.urlBase}/signalk/v2/api/resources/regions/${regionGUID}`)


@panaaj do we have a way to call the resources api from a plugin, to skip the http call?

src/index.ts

tkurki · 2025-10-27T18:59:27Z

src/index.ts

+      const { jobId, downloader } = ChartDownloader.createAndRegister(urlBase, defaultChartsPath, provider)
+      // Long going process, create an endpoint to monitor progress
+      ;(async () => {
+          await downloader.deleteTiles(regionGUID)


This seems like a weird construction. Can't deleteTiles start deleting the files in the background and return a Promise, and here we just don't await it?

Tried to tidy this up a little

src/index.ts

batuakan · 2025-10-28T14:56:27Z

This mostly worked, except when I entered a non working chart source the result was this:

This feature is a bit dangerous, as it can fill up your disk and effectively lock people out of their system if the system won't start properly with disk full.

I propose at least the following safeguards:
* minimum free disk space - so that seeding stops if there is not enough free space

* some threshold for seeding region size, maybe with just an alert to confirm that the region is large
This kind of long running operations easily error out for one reason or another, and you don't want to start over from the very beginning. The jobs are not persisted over restarts, so it would be useful if the UI could present a list of cached providers and their seeding statuses: what regions are seeded and their status, for example in terms of number of files on disk and a [Continue seeding] button, that would create the tiles list, check for their existence and fetch only missing tiles.

I have added a safeguard for diskspace, for the moment if disk space runs lower than 1GB you cannot seed anymore

For the second safeguard, I am thinking of changing the code a little bit, After selecting chart provider and region and click create, it will just create the job instance, calculate the total number tiles, cached tiles and number of tiles to download and show them on webapp, then the user can start actual seeding or deleting process, We can here also add if the number of tiles is greater than lets say 1 million, we can show another confirmation box, saying that is a lot of tiles, consider seeding smaller regions.

Since nobody likes regions, i am also considering /cache/:identifier endpoint to accept several types of inputs, e.g. regionGUID, tile [z,x,y] and all of its subtiles until maxzoom, and bbox. I guess that provides more flexible api and more suitable for different use cases

Trying to fix the anti-meridian thingy now

…rs to pass to the remote tile server

- antimeridian check - Added support to seed regions, bboz or tile and subtiles - Updated webui - Added safeguards for low diskspace and huge downloads - Code restructuring - Changed signalk typo to SignalK

tkurki reviewed Oct 20, 2025

View reviewed changes

tkurki reviewed Oct 27, 2025

View reviewed changes

batuakan added 7 commits November 1, 2025 05:55

Added support to cache online tile maps, also added custom http heade…

60267b9

…rs to pass to the remote tile server

Added chart downloader web addin for charts-addin

c2408a2

Updated readme and changed config parameter name from cached to proxy

b677618

Fixed failing test

1e10958

Added ~tilePath~ instead of full url

8915b1e

PR fixes

67ea7ca

- antimeridian check - Added support to seed regions, bboz or tile and subtiles - Updated webui - Added safeguards for low diskspace and huge downloads - Code restructuring - Changed signalk typo to SignalK

Small fix for header parsing

f16dd19

batuakan force-pushed the caching_online_map_sources branch from 37dba5a to f16dd19 Compare November 1, 2025 04:55

tkurki added the feature label Nov 1, 2025

Fixed build issue after merge

2d6b019

Added support to cache online tile maps, also added custom http heade… #41

Are you sure you want to change the base?

Added support to cache online tile maps, also added custom http heade… #41

Uh oh!

Conversation

batuakan commented Oct 20, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tkurki left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

batuakan commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

batuakan commented Oct 28, 2025 •

edited

Loading