Skip to content

Make tile extent runtime configurable with default 4096#1565

Open
tasiotas wants to merge 3 commits into
onthegomap:mainfrom
tasiotas:tile-extent-runtime-config
Open

Make tile extent runtime configurable with default 4096#1565
tasiotas wants to merge 3 commits into
onthegomap:mainfrom
tasiotas:tile-extent-runtime-config

Conversation

@tasiotas

Copy link
Copy Markdown

Hi

I added runtime arg for tile extent. Defaults to 4096, I tested it with 8192 and 16384. My use case required higher precision because I'm rendering maps with tileserver-gl at very high dpi for print.

Passed all tests. I used Codex 5.3

@github-actions

github-actions Bot commented May 14, 2026

Copy link
Copy Markdown
This Branch d797c1a Base 1a69e5e
0:01:10 DEB [archive] - Tile stats:
0:01:10 DEB [archive] - Biggest tiles (gzipped)
1. 14/4942/6092 (162k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.40015 (poi:89k)
2. 9/154/190 (149k) https://onthegomap.github.io/planetiler-demo/#9.5/41.77078/-71.36719 (landcover:86k)
3. 10/308/381 (138k) https://onthegomap.github.io/planetiler-demo/#10.5/41.63994/-71.54297 (landcover:72k)
4. 10/308/380 (137k) https://onthegomap.github.io/planetiler-demo/#10.5/41.90214/-71.54297 (landcover:66k)
5. 14/4941/6092 (120k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.42212 (poi:69k)
6. 14/4941/6093 (118k) https://onthegomap.github.io/planetiler-demo/#14.5/41.81227/-71.42212 (poi:62k)
7. 14/4946/6112 (111k) https://onthegomap.github.io/planetiler-demo/#14.5/41.50035/-71.31226 (building:67k)
8. 14/4946/6113 (110k) https://onthegomap.github.io/planetiler-demo/#14.5/41.48389/-71.31226 (building:59k)
9. 14/4940/6092 (102k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.44409 (building:92k)
10. 14/4942/6091 (100k) https://onthegomap.github.io/planetiler-demo/#14.5/41.84501/-71.40015 (building:79k)
0:01:10 DEB [archive] - Max tile sizes
                      z0    z1    z2    z3    z4    z5    z6    z7    z8    z9   z10   z11   z12   z13   z14   all
           boundary  151   337   410   542   803   290   394   492   672  1.6k    2k  6.9k  6.2k  5.6k  4.4k  6.9k
              water 7.7k  3.7k  8.6k  5.5k  2.6k  5.1k   15k   18k   16k   26k   15k   13k   17k   15k   12k   26k
              place    0     0   487   487   487   733   822  1.1k  1.8k  3.3k  6.2k  3.8k    2k   935    1k  6.2k
            landuse    0     0     0     0   548   693  1.6k    7k   18k   44k   58k   49k   38k   19k   12k   58k
     transportation    0     0     0     0   354    1k  1.5k  4.5k  6.3k   21k   15k   17k   67k   38k   38k   67k
           waterway    0     0     0     0   112   119     0     0     0  3.3k  2.4k  2.1k  2.1k  4.9k  2.4k  4.9k
               park    0     0     0     0     0     0  1.1k    4k  9.6k   19k   13k  8.2k  3.7k  3.4k  4.4k   19k
transportation_name    0     0     0     0     0     0   293   360  1.1k  1.9k  5.8k  4.8k    4k  3.5k   18k   18k
          landcover    0     0     0     0     0     0     0  9.7k   29k   86k   72k   82k   53k   30k   25k   86k
      mountain_peak    0     0     0     0     0     0     0  1.1k  1.8k  3.4k  4.4k  2.8k  1.4k  1.4k   869  4.4k
         water_name    0     0     0     0     0     0     0     0     0   528   503   475   494  1.2k  1.5k  1.5k
    aerodrome_label    0     0     0     0     0     0     0     0     0     0   666   289   273   221   221   666
            aeroway    0     0     0     0     0     0     0     0     0     0  1.6k    2k    3k  3.3k  2.8k  3.3k
                poi    0     0     0     0     0     0     0     0     0     0     0     0   589   586   89k   89k
           building    0     0     0     0     0     0     0     0     0     0     0     0     0   60k   92k   92k
        housenumber    0     0     0     0     0     0     0     0     0     0     0     0     0     0   35k   35k
          full tile 7.9k    4k  9.5k  6.5k  3.7k  6.3k   21k   41k   85k  202k  185k  135k  114k  120k  255k  255k
            gzipped 6.2k  3.5k  7.1k  5.2k  3.1k    5k   14k   29k   60k  149k  138k   99k   84k   85k  162k  162k
0:01:10 DEB [archive] -    Max tile: 255k (gzipped: 162k)
0:01:10 DEB [archive] -    Avg tile: 5.5k (gzipped: 4.1k) using weighted average based on OSM traffic
0:01:10 DEB [archive] -     # tiles: 4,115,030
0:01:10 DEB [archive] -  # features: 5,772,614
0:01:10 INF [archive] - Finished in 21s cpu:1m17s avg:3.7
0:01:10 INF [archive] -   read    1x(3% 0.6s wait:19s done:1s)
0:01:10 INF [archive] -   encode  4x(54% 11s wait:2s done:1s)
0:01:10 INF [archive] -   write   1x(20% 4s wait:14s done:1s)
0:01:10 INF [archive] - Finished in 1m11s cpu:3m38s gc:1s avg:3.1
0:01:10 INF [archive] - FINISHED!
0:01:10 INF [archive] - 
0:01:10 INF [archive] - ----------------------------------------
0:01:10 INF [archive] - data errors:
0:01:10 INF [archive] - 	render_snap_fix_input	16,807
0:01:10 INF [archive] - 	osm_multipolygon_missing_way	396
0:01:10 INF [archive] - 	osm_boundary_missing_way	55
0:01:10 INF [archive] - 	merge_snap_fix_input	9
0:01:10 INF [archive] - 	feature_polygon_osm_invalid_multipolygon_empty_after_fix	8
0:01:10 INF [archive] - 	osm_multipolygon_duplicate_member	4
0:01:10 INF [archive] - 	feature_centroid_if_convex_osm_invalid_multipolygon_empty_after_fix	2
0:01:10 INF [archive] - 	omt_fix_water_before_ne_intersect	2
0:01:10 INF [archive] - 	feature_point_on_surface_osm_invalid_multipolygon_empty_after_fix	1
0:01:10 INF [archive] - 	render_snap_fix_input2	1
0:01:10 INF [archive] - 	omt_park_area_osm_invalid_multipolygon_empty_after_fix	1
0:01:10 INF [archive] - 	omt_water_osm_invalid_multipolygon_empty_after_fix	1
0:01:10 INF [archive] - ----------------------------------------
0:01:10 INF [archive] - 	overall          1m11s cpu:3m38s gc:1s avg:3.1
0:01:10 INF [archive] - 	lake_centerlines 3s cpu:7s avg:2.3
0:01:10 INF [archive] - 	  read     1x(17% 0.5s done:2s)
0:01:10 INF [archive] - 	  process  4x(0% 0s done:2s)
0:01:10 INF [archive] - 	  write    1x(0% 0s done:2s)
0:01:10 INF [archive] - 	water_polygons   15s cpu:39s avg:2.5
0:01:10 INF [archive] - 	  read     1x(44% 7s done:7s)
0:01:10 INF [archive] - 	  process  4x(22% 3s wait:5s done:6s)
0:01:10 INF [archive] - 	  write    1x(3% 0.4s wait:9s done:5s)
0:01:10 INF [archive] - 	natural_earth    11s cpu:19s avg:1.7
0:01:10 INF [archive] - 	  read     1x(58% 6s done:5s)
0:01:10 INF [archive] - 	  process  4x(8% 0.9s wait:6s done:5s)
0:01:10 INF [archive] - 	  write    1x(0% 0s wait:7s done:5s)
0:01:10 INF [archive] - 	osm_pass1        2s cpu:7s avg:3.3
0:01:10 INF [archive] - 	  read     1x(2% 0s wait:2s)
0:01:10 INF [archive] - 	  parse    4x(36% 0.8s)
0:01:10 INF [archive] - 	  process  1x(68% 1s)
0:01:10 INF [archive] - 	osm_pass2        17s cpu:1m6s avg:4
0:01:10 INF [archive] - 	  read     1x(0% 0s wait:10s done:6s)
0:01:10 INF [archive] - 	  process  4x(69% 11s)
0:01:10 INF [archive] - 	  write    1x(3% 0.4s wait:16s)
0:01:10 INF [archive] - 	ne_lakes         0s cpu:0s avg:0
0:01:10 INF [archive] - 	boundaries       0s cpu:0s avg:0
0:01:10 INF [archive] - 	agg_stop         0s cpu:0s avg:0
0:01:10 INF [archive] - 	sort             1s cpu:3s avg:2.5
0:01:10 INF [archive] - 	  worker  1x(52% 0.7s)
0:01:10 INF [archive] - 	archive          21s cpu:1m17s avg:3.7
0:01:10 INF [archive] - 	  read    1x(3% 0.6s wait:19s done:1s)
0:01:10 INF [archive] - 	  encode  4x(54% 11s wait:2s done:1s)
0:01:10 INF [archive] - 	  write   1x(20% 4s wait:14s done:1s)
0:01:10 INF [archive] - ----------------------------------------
0:01:10 INF [archive] - 	archive	109MB
0:01:10 INF [archive] - 	features	297MB
-rw-r--r-- 1 runner runner 87M May 18 14:54 run.jar
0:01:06 DEB [archive] - Tile stats:
0:01:06 DEB [archive] - Biggest tiles (gzipped)
1. 14/4942/6092 (162k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.40015 (poi:89k)
2. 9/154/190 (149k) https://onthegomap.github.io/planetiler-demo/#9.5/41.77078/-71.36719 (landcover:86k)
3. 10/308/381 (138k) https://onthegomap.github.io/planetiler-demo/#10.5/41.63994/-71.54297 (landcover:72k)
4. 10/308/380 (137k) https://onthegomap.github.io/planetiler-demo/#10.5/41.90214/-71.54297 (landcover:66k)
5. 14/4941/6092 (121k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.42212 (poi:69k)
6. 14/4941/6093 (118k) https://onthegomap.github.io/planetiler-demo/#14.5/41.81227/-71.42212 (poi:62k)
7. 14/4946/6112 (111k) https://onthegomap.github.io/planetiler-demo/#14.5/41.50035/-71.31226 (building:67k)
8. 14/4946/6113 (110k) https://onthegomap.github.io/planetiler-demo/#14.5/41.48389/-71.31226 (building:59k)
9. 14/4940/6092 (102k) https://onthegomap.github.io/planetiler-demo/#14.5/41.82864/-71.44409 (building:92k)
10. 14/4942/6091 (100k) https://onthegomap.github.io/planetiler-demo/#14.5/41.84501/-71.40015 (building:79k)
0:01:06 DEB [archive] - Max tile sizes
                      z0    z1    z2    z3    z4    z5    z6    z7    z8    z9   z10   z11   z12   z13   z14   all
           boundary  151   336   409   544   802   287   396   490   670  1.6k    2k  6.9k  6.2k  5.6k  4.4k  6.9k
              water 7.7k  3.7k  8.6k  5.5k  2.6k  5.1k   15k   18k   16k   26k   15k   13k   17k   15k   12k   26k
              place    0     0   487   487   487   733   822  1.1k  1.8k  3.3k  6.2k  3.8k    2k   935    1k  6.2k
            landuse    0     0     0     0   549   695  1.6k    7k   18k   44k   58k   49k   38k   19k   12k   58k
     transportation    0     0     0     0   355    1k  1.5k  4.5k  6.4k   21k   15k   17k   67k   38k   38k   67k
           waterway    0     0     0     0   112   119     0     0     0  3.3k  2.4k  2.1k  2.1k  4.9k  2.4k  4.9k
               park    0     0     0     0     0     0  1.1k    4k  9.6k   19k   13k  8.2k  3.7k  3.4k  4.4k   19k
transportation_name    0     0     0     0     0     0   293   360  1.1k  1.9k  5.8k  4.8k    4k  3.5k   18k   18k
          landcover    0     0     0     0     0     0     0  9.7k   29k   86k   72k   82k   53k   30k   25k   86k
      mountain_peak    0     0     0     0     0     0     0  1.1k  1.8k  3.4k  4.4k  2.8k  1.4k  1.4k   869  4.4k
         water_name    0     0     0     0     0     0     0     0     0   528   503   475   494  1.2k  1.5k  1.5k
    aerodrome_label    0     0     0     0     0     0     0     0     0     0   666   289   273   221   221   666
            aeroway    0     0     0     0     0     0     0     0     0     0  1.6k    2k    3k  3.3k  2.8k  3.3k
                poi    0     0     0     0     0     0     0     0     0     0     0     0   589   586   89k   89k
           building    0     0     0     0     0     0     0     0     0     0     0     0     0   59k   92k   92k
        housenumber    0     0     0     0     0     0     0     0     0     0     0     0     0     0   35k   35k
          full tile 7.9k    4k  9.5k  6.5k  3.7k  6.3k   21k   41k   85k  202k  185k  135k  114k  120k  255k  255k
            gzipped 6.2k  3.5k  7.1k  5.2k  3.1k    5k   14k   29k   60k  149k  138k   99k   84k   85k  162k  162k
0:01:06 DEB [archive] -    Max tile: 255k (gzipped: 162k)
0:01:06 DEB [archive] -    Avg tile: 5.5k (gzipped: 4.1k) using weighted average based on OSM traffic
0:01:06 DEB [archive] -     # tiles: 4,115,030
0:01:06 DEB [archive] -  # features: 5,772,614
0:01:06 INF [archive] - Finished in 21s cpu:1m17s avg:3.7
0:01:06 INF [archive] -   read    1x(3% 0.6s wait:19s done:1s)
0:01:06 INF [archive] -   encode  4x(54% 11s wait:2s done:1s)
0:01:06 INF [archive] -   write   1x(19% 4s wait:15s done:1s)
0:01:06 INF [archive] - Finished in 1m6s cpu:3m36s gc:1s avg:3.2
0:01:06 INF [archive] - FINISHED!
0:01:06 INF [archive] - 
0:01:06 INF [archive] - ----------------------------------------
0:01:06 INF [archive] - data errors:
0:01:06 INF [archive] - 	render_snap_fix_input	16,807
0:01:06 INF [archive] - 	osm_multipolygon_missing_way	396
0:01:06 INF [archive] - 	osm_boundary_missing_way	55
0:01:06 INF [archive] - 	merge_snap_fix_input	9
0:01:06 INF [archive] - 	feature_polygon_osm_invalid_multipolygon_empty_after_fix	8
0:01:06 INF [archive] - 	osm_multipolygon_duplicate_member	4
0:01:06 INF [archive] - 	feature_centroid_if_convex_osm_invalid_multipolygon_empty_after_fix	2
0:01:06 INF [archive] - 	omt_fix_water_before_ne_intersect	2
0:01:06 INF [archive] - 	feature_point_on_surface_osm_invalid_multipolygon_empty_after_fix	1
0:01:06 INF [archive] - 	render_snap_fix_input2	1
0:01:06 INF [archive] - 	omt_park_area_osm_invalid_multipolygon_empty_after_fix	1
0:01:06 INF [archive] - 	omt_water_osm_invalid_multipolygon_empty_after_fix	1
0:01:06 INF [archive] - ----------------------------------------
0:01:06 INF [archive] - 	overall          1m6s cpu:3m36s gc:1s avg:3.2
0:01:06 INF [archive] - 	lake_centerlines 2s cpu:6s avg:2.5
0:01:06 INF [archive] - 	  read     1x(21% 0.5s done:2s)
0:01:06 INF [archive] - 	  process  4x(0% 0s done:2s)
0:01:06 INF [archive] - 	  write    1x(0% 0s done:2s)
0:01:06 INF [archive] - 	water_polygons   15s cpu:37s avg:2.5
0:01:06 INF [archive] - 	  read     1x(44% 7s done:7s)
0:01:06 INF [archive] - 	  process  4x(22% 3s wait:5s done:6s)
0:01:06 INF [archive] - 	  write    1x(3% 0.4s wait:9s done:5s)
0:01:06 INF [archive] - 	natural_earth    7s cpu:14s avg:2.1
0:01:06 INF [archive] - 	  read     1x(96% 6s)
0:01:06 INF [archive] - 	  process  4x(13% 0.9s wait:6s)
0:01:06 INF [archive] - 	  write    1x(0% 0s wait:7s)
0:01:06 INF [archive] - 	osm_pass1        2s cpu:8s avg:3.5
0:01:06 INF [archive] - 	  read     1x(2% 0s wait:2s)
0:01:06 INF [archive] - 	  parse    4x(34% 0.8s)
0:01:06 INF [archive] - 	  process  1x(70% 2s)
0:01:06 INF [archive] - 	osm_pass2        17s cpu:1m9s avg:4
0:01:06 INF [archive] - 	  read     1x(0% 0s wait:11s done:6s)
0:01:06 INF [archive] - 	  process  4x(68% 12s)
0:01:06 INF [archive] - 	  write    1x(3% 0.4s wait:17s)
0:01:06 INF [archive] - 	ne_lakes         0s cpu:0s avg:0
0:01:06 INF [archive] - 	boundaries       0s cpu:0s avg:0
0:01:06 INF [archive] - 	agg_stop         0s cpu:0s avg:0
0:01:06 INF [archive] - 	sort             1s cpu:4s avg:2.6
0:01:06 INF [archive] - 	  worker  1x(48% 0.7s)
0:01:06 INF [archive] - 	archive          21s cpu:1m17s avg:3.7
0:01:06 INF [archive] - 	  read    1x(3% 0.6s wait:19s done:1s)
0:01:06 INF [archive] - 	  encode  4x(54% 11s wait:2s done:1s)
0:01:06 INF [archive] - 	  write   1x(19% 4s wait:15s done:1s)
0:01:06 INF [archive] - ----------------------------------------
0:01:06 INF [archive] - 	archive	109MB
0:01:06 INF [archive] - 	features	297MB
-rw-r--r-- 1 runner runner 87M May 18 14:55 run.jar

Full logs: https://github.com/onthegomap/planetiler/actions/runs/26041130395

@msbarry msbarry left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this! I'm a big nervous about other places that have a 4096 assumption built into them without explicitly referring to 4096. I think the end to end tests in PlanetilerTests will help flush out any of those.

Comment on lines +94 to +96
if (tileExtent <= 0) {
throw new IllegalArgumentException("tile_extent must be > 0, was " + tileExtent);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that extent needs to be a power of 2?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, added power of 2 check

), "layer", Map.of(), 0)
)
)), sortListValues(results.tiles));
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test for end to end point/line/polygon for a few different values of tile extent? At least 8192 and 16384, possibly a lower value like 512 or 1024 as well? I think we should probably test a worst case for each of those with coordinates at (-255, -255) (513, -255) (513, 513) (-255, 513) and a few points in the middle

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

scale = Math.min(31 - 14, scale);
long maxCoordinate = config.tileExtent() * 4L;
int bits = 64 - Long.numberOfLeadingZeros(maxCoordinate - 1);
scale = Math.clamp(scale, 0, Math.max(0, 31 - bits));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few more subtle places that the 4096 assumption has snuck in that might not explicitly reference 4096 - this is one of them, thanks for fixing! I'm trying to think if there might be any others...

int y = zigZagEncode((int) Math.round(coord.y * 4096 / 256));
int x = zigZagEncode((int) Math.round(coord.x * tileExtent / SIZE));
int y = zigZagEncode((int) Math.round(coord.y * tileExtent / SIZE));
return (int) Hilbert.hilbertXYToIndex(15, x, y);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Zig-zag encoding maps coordinates in a range from -2^14 -> 2^14 to 0 -> 2^15. That assumption was based off 4096x4096 tiles with a 4096 buffer, so -4096 -> 8192 that gets mapped to 0 -> 16384 (2^14 plus 1 for safety). I think we could probably bump 15 up to 16, but then if tile extent / size exceeds a certain threshold then we might need to shift x and y right by a certain amount to keep it in the right range.

Comment on lines +52 to +56
/**
* Rounding precision for 256x256px tiles encoded using {@code tile_extent} values (default 4096).
*/
private static final AtomicReference<PrecisionModel> tilePrecision =
new AtomicReference<>(new PrecisionModel(4096d / 256d));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we move all references to 4096 out of static variables and either pass them as args to functions that need them or extract them from geoutils to a class that you instantiate with a tile extent. That might make this PR very big though, let me know what you think - if it's too much I could do in a followup PR.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This issue also asks for the extent to be configurable per-zoom #1286 so we might even want to make it a global setting 🤔

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how would you implement per zoom tile extent configuration? per zoom args?
--tileExtentZ12, --tileExtentZ13, etc... ?

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants