OSM Tileserver

Historic information about this is on OSM Tileserver/Archive.

This page is for the project to implement an OpenStreetMap TileServer hosted in WMF's production infrastructure which serves the same basic data as OSM themselves (synced from them). This is orthogonal to (or in some other sense, a prerequisite for) other OSM efforts going on in Labs for related overlays and databases. Corrections to the information below are welcome, this is all to the best of my knowledge at this time...

• https://phabricator.wikimedia.org/T204506
• https://tools.wmflabs.org/openstack-browser/project/maps
• https://horizon.wikimedia.org
• maps-tiles1.maps.eqiad.wmflabs THIS IS THE NEW DEBIAN STRETCH HOST
• maps-tiles2.maps.eqiad.wmflabs
  • This is renderd slave 1
• maps-tiles3.maps.eqiad.wmflabs
  • This is renderd master
  • This server is the only webhost

• Apache with mod_tile and mod rewrite
  • conf /etc/apache2/sites-enabled/tileserver_site.conf
  • https://launchpad.net/~osmadmins/+archive/ubuntu/ppa?field.series_filter=bionic
  • lib: /var/lib/mod_tile
    • deps for compiling: apt-get install build-essential autoconf libtool libmapnik-dev apache2-dev
    • docs for running mod_tile on jessie
    • /usr/lib/apache2/modules/mod_tile.so
  • puppet: include ::apache ?
• Munin: Create graphs of tile freshness https://tiles.wmflabs.org/munin/mod_tile-month.html
  • conf:
  • output: /var/www/munin
  • apt: sudo apt-get install munin munin-node munin-plugins-core munin-plugins-extra
• Renderd
  • conf: /etc/renderd.conf
  • output: /data/project/tiles/[tilevariant]
  • Important: make sure fonts for global scripts are installed.
  • Hillshading is not rendered it seems. pre generated tiles ?
  • apt: renderd/trusty
  • renderd, can have slaves. this is determined by the --slave param when starting the renderd daemon
• Mapnik
  • Version: 2.2.0
  • Styles: /data/project/styles/
  • Lib: /usr/lib/mapnik/2.2/input
  • apt: libmapnik-dev/trusty, libmapnik2.2/trusty, mapnik-utils/trusty, openstreetmap-mapnik-carto-stylesheet-data/trusty, python-mapnik/trusty
    • Stretch: libmapnik-dev, libmapnik3.0, mapnik-utils, python3-mapnik
• Database
  • Host: osmdb.eqiad.wmnet
  • User: osm
  • Database: gis
  • Referenced from /data/project/styles/*/*.xml
• OSM Stylesheets
  • Seems most of these are not versioned, have not been updated since 2014 and we seem to be missing the .mml (we only have the output)
  • likely will have to keep using what we already have generated for now.
  • apt: openstreetmap-carto
  • Hikebike: hikebikemap-carto
  • OSM Default: openstreetmap-carto
  • OSM Default no labels:
  • bw-mapnik:
• Installed fonts:
  • fonts-dejavu, fonts-dejavu-core, fonts-dejavu-extra, fonts-noto fonts-khmeros, fonts-khmeros-core, font-sil-padauk, fonts-sipa-arundina, ttf-indic-fonts-core, ttf-kannada-fonts, ttf-tamil-fonts, xfonts-encodings, xfonts-utils
  • stretch: fonts-dejavu fonts-dejavu-core fonts-dejavu-extra (削除) fonts-droid (削除ここまで) fonts-khmeros (削除) fonts-khmeros-core (削除ここまで) (削除) font-sil-padauk (削除ここまで) fonts-sipa-arundina (削除) ttf-indic-fonts-core (削除ここまで) xfonts-encodings xfonts-utilsfonts-taml fonts-samyak-taml fonts-lohit-taml fonts-lohit-knda fonts-knda
  • puppet: include ::mediawiki::packages::fonts
• postgresql-client for debugging

• Can we puppetize this ?
• Should we switch from renderd to tirex ?
• Where does the code for renderd come from ?
• Why is maps-tiles2.maps.eqiad.wmflabs not serving http ?
• Is there no osm2pqsl ??????????
• The styles are 83MB.. but contain the dbname etc. How to not put that in puppet repo, yet have dbname configurable ?
• what is the difference between the https and the apache manifests ?
• how to force re-rendering of tiles. See this.

• Dump munin, since grafana and nagf provide the same info
• Build a new cluster on hostname maps.wmflabs.org (currently unused), for easy testing
• Switch to Debian Stretch as best supported
• Create a repo to maintain all the styles, config files and any non-privileged information
• Get rid of some no longer active wmflabs maintainers from the project
• delete overpass-wiki instance. Made by User:Jotpe, but doesn't seem to be in actual use.
• Make a debian stretch package of mod_tile ??????? See: APT_repository