Hi,
I'm just beginning my usage of Open Railway Map - thank you so much :) As a thank you
for all the hard work you do I donated 100 EUR as well.
I've successfully setup everything on my machine and I'm now trying to generate
complete file-based maps[1] up to zoom level 15 for Sweden. I've written a
"offline renderer" using the node-tileserver that accepts a rectangular range of
co-ordinates (i.e only Sweden) and iteratively generates the tiles. Generating the 715 378
unique tiles (x4 for all bitmap variants and json) takes roughly a week using 8 cores and
52GB of ram. I'm guessing I'm doing something super inefficient.
- Machine: Google Cloud VM, 8 CPU with 52GB ram plus 200 GB of SSD disk
- Only loaded Europe, not entire planet.
- Postgresql:
- 12GB shared buffer
- 1.5GB work_mem
- autovacuum = Off
The node worker is using Redis and Bull (
https://github.com/OptimalBits/bull
<
https://github.com/OptimalBits/bull>) and is multi-core. It's a fairly simple
version[2] (see source code), but I'm lacking quite a bit understanding of which parts
of node-tileserver I need to use so it's a bit of a hack.
My best guess is that it's the database queries that are slow I need to do additional
tuning. Since the DB is only 6GB for Europe I'm thinking about moving Postgresql to
ramdisk.
Any helpful pointers? I've looked into the Postgresql tuning guides (both general and
PostGIS). So far I'm only getting a few tiles per second.
If you'r interested in the code itself it's available on
https://gist.github.com/bivald/975e56a0850408255cbb6fc9e921df63
<
https://gist.github.com/bivald/975e56a0850408255cbb6fc9e921df63>
I'm fluent in JS and Python if that helps.
Regards,
Niklas
[1] The reason is that since we only want to cover small parts of the world, the hosting
for generated maps is so much simpler then hosting API:s and TileServers - not to mention
faster.
[2]
https://gist.github.com/bivald/975e56a0850408255cbb6fc9e921df63
<
https://gist.github.com/bivald/975e56a0850408255cbb6fc9e921df63>