summary

This post shows how to use open software to

• create vector tiles using openmaptiles open source software
• serve those vector files with a very short jsnode program
• display those vector tiles with a very short Mapbox GL JS based app

Vector tiles are a potentially faster way than raster tiles for map tiles servers to send tile information to client apps. It is faster because the polygons and symbols of the vector format offer greater compression than can be achieved with raster format. Also, decisions such as which language to render can be made on the client end.

Of course that requires the client end to have a fast way to render the vector tiles. The free and open source Mapbox GL JS software provided by Mapbox fulfills that requirement.

However, there is no completely free source of vector tiles available for personal or business use. Fortunately, through the efforts of OpenSourceMap (OSM) and OSM Carto open source data for maps and open source software for building and displaying custom maps is available.

Mapbox introduced and made available as open source their Mapbox GS JS software capable of high speed rendering of vector tiles.

An effort (not under the aupices of OSM or OSM Carto) to build upon the work of OSM Carto and provide a convenient tool chain to create vector tiles compatible with Mapbox GS JS was made under the name osm2vectortiles. Unfortunately, rather than developing tiles which resemble the OSM Carto (raster) tile style, they ended up too closely resembling Mapbox‘s proprietary Mapbox Streets tile style. So the osm2vectortiles project was voluntarily shutdown.

“The roadmap to hell is paved with good intentions”

The non-Mapbox-proprietary parts of osm2vectortiles were adopted into a new open source project openmaptiles. A new style for vectors tiles was created which doesn’t resemble Mapbox‘s proprietary styles, but can still be used with Mapbox GL JS rendering software.

The very first part of this post describes creating tiles using the openmaptiles software. The parts thereafter describe serving tiles and displaying those served tiles in an app.

Unfortunately the openmaptiles github and openmaptiles.org sites do not offer simple straightforward minimalist examples showing how to serve tiles, and how to display served tiles in an application.

There is a serving program tileserver-gl associated with openmaptiles but it is overly complex for a newcomer who want to prepare a minimalist server.

There is an example for displaying vector tiles from a Mapbox GL JS app, but that is set up to get its tile data from a commercial site, rather than self hosted tiles.

So this post bridges the gap and describes a minimalist framework to serve and display self hosted vector openmaptiles tiles. It’s not really difficult, but there are a few frustrating gotchas which this post aims to make easier.

Note that this post keeps the tileserver and display app seperate served in order to test cross-origin conditions.

Creating the tiles with openmaptiles

Clone the openmaptiles github project. We’ll refer to the resulting directory as $OMT. From the directory above where $OMT will be:

git clone https://github.com/openmaptiles/openmaptiles
cd $OMT

Make sure the required software is installed as explained in the openmaptiles instructions.

Download the file SanFrancisco.osm.pbf. Put it in the folder $OMT/data.

curl -o data/SanFrancisco.osm.pbf \
 https://download.bbbike.org/osm/bbbike/SanFrancisco/SanFrancisco.osm.pbf

Modify the following files:

• $OMT/docker-compose.yml
• $OMT/.env

to have BBOX and MAX_ZOOM values as follows:

BBOX: " -122.54, 37.54, -122.32, 37.93"
MAX_ZOOM: "14"

Create the file $OMT/data/docker-compose-config.yml with content

version: "2"
services:
  generate-vectortiles:
    environment:
      BBOX: " -122.54, 37.54, -122.32, 37.93"
      OSM_MAX_TIMESTAMP : "2018-06-15T17:15:4300:00:00Z"
      OSM_AREA_NAME: "SanFrancisco"
      MIN_ZOOM: "0"
      MAX_ZOOM: "14"

From the $OMT directory, execute

./quickstart.sh SanFrancisco

After a long while, the tiles will be created in the file $OTM/data/tiles.mbtiles. From the $OMT directory run

make start-tileserver

That will a docker module running tileserver-gl. Then open your browser to

http://localhost:8080

to see the tile data.

The program ‘tileserver-gl’ is too complex to qualify as a minimalist server. To close the tileserver-gl program and remove its docker image you may need commands such as:

docker image ls
docker image rm <image ID number>

Minimalist tile server

The tile server will serve data from the ‘tiles.mbtiles’ data, sending it via http to the tile display app. The tile server will also serve the fonts and sprites necessary for drawing the map. That is all the data that the app requires to display the map.

Set up fonts and sprites

The fonts are available from the github openmaptiles project.

Let $FND be a directory which will hold the fonts. The last directory will be named fonts as a result of the clone process. Clone and execute as follows:

git clone git@github.com:openmaptiles/fonts.git
cd $FND
npm install
node ./generate.js

Finally the static font files will be ready under ‘$FND/_output`.

Add the sprites here too because, why not?:

cd $FND
mkdir sprite
curl -o sprite/sprite.png https://openmaptiles.github.io/osm-bright-gl-style/sprite.png

Minimalist tile/font/sprite server

Create a directory for the minimalist tile server. We’ll call that directory $MTS.

Create the following file $MTS/index.js, (borrowed in part from this Git GIST by manuelroth):

var express = require("express"),
    app = express(),
    MBTiles = require('mbtiles'),
    p = require("path");


app.use('/fonts', express.static('fonts', {
    setHeaders: function setHeaders(res, path, stat) {
        res.header('Access-Control-Allow-Origin', '*');
        res.header('Access-Control-Allow-Methods', 'GET');
        res.header('Access-Control-Allow-Headers', 'Content-Type');
    }
}))

app.use('/sprite', express.static('sprite', {
    setHeaders: function setHeaders(res, path, stat) {
        res.header('Access-Control-Allow-Origin', '*');
        res.header('Access-Control-Allow-Methods', 'GET');
        res.header('Access-Control-Allow-Headers', 'Content-Type');
    }
}))


// Enable CORS and set correct mime type/content encoding
var header = {
  "Access-Control-Allow-Origin":"*",
  "Access-Control-Allow-Headers":"Origin, X-Requested-With, Content-Type, Accept",
  "Content-Type":"application/x-protobuf",
  "Content-Encoding":"gzip"
};

// Route which handles requests like the following: /<mbtiles-name>/0/1/2.pbf
app.get('/:source/:z/:x/:y.pbf', function(req, res) {
    console.log('req.params.source : ' + req.params.source)
    console.log('MBTiles file : ' +  p.join(__dirname, req.params.source + '.mbtiles'))
    console.log(req.params.z + '/' + req.params.x + '/' + req.params.y)
  new MBTiles(p.join(__dirname, req.params.source + '.mbtiles'), function(err, mbtiles) {
    mbtiles.getTile(req.params.z, req.params.x, req.params.y, function(err, tile, headers) {
      if (err) {
        res.set({"Content-Type": "text/plain"});
        res.status(404).send('Tile rendering error: ' + err + '\n');
      } else {
        res.set(header);
        res.send(tile);
      }
    });
    if (err) console.log("error opening database");
  });
});

// Starts up the server on port 3000
console.log('__dirname : ' + __dirname)
console.log('Listening on port: ' + 3000);
app.listen(3000);

Install under $MTS the necessary node modules:

npm install express --save
npm install mbtiles --save

Create a symbolic link to the tiles file

ln -s $OMT/data/tiles.mbtiles $MTS/tiles.mbtiles

Create a symbolic link to the fonts

ln -s $FND/_output $MTS/fonts

Create a symbolic link to the sprites directory

ln -s $FND/sprite $MTS/sprite

Minimalist display app: MapboxGLJS with 3rd party vector tiles

Create a directory for the front end app. We’ll call it $MGJ

Style file

Download the style file osm-bright-gl-style.json from the openmaptiles project:

cd $MGJ
curl -o osm-bright-gl-style.json \
 https://raw.githubusercontent.com/openmaptiles/osm-bright-gl-style/master/style.json

Backup the original:

cp osm-bright-gl-style.json osm-bright-gl-style.json.orig

Modify osm-bright-gl-style.json as follows:

"sources": {
  "openmaptiles": {
    "type": "vector",
    "url": "v3.json"
  }
},
"sprite": "https://openmaptiles.github.io/osm-bright-gl-style/sprite",
"glyphs": "https://free.tilehosting.com/fonts/{fontstack}/{range}.pbf?key={key}",

"sources": {
  "openmaptiles": {
    "type": "vector",
      "tiles":["http://localhost:3000/tiles/{z}/{x}/{y}.pbf"],
      "maxzoom": 14
  }
},
  "sprite": "http://localhost:3000/sprite/sprite",
  "glyphs": "http://localhost:3000/fonts/{fontstack}/{range}.pbf",

Simple Mapbox GL JS (with 3rd party tiles) app

Create the file index.html in the $MGJ directory

<!DOCTYPE html>
<html>
<head>
    <meta charset='utf-8' />
    <title>OpenMapTiles OSM Bright style</title>
    <meta name='viewport' content='initial-scale=1,maximum-scale=1,user-scalable=no' />
    <script src='https://api.tiles.mapbox.com/mapbox-gl-js/v0.29.0/mapbox-gl.js'></script>
    <link href='https://api.tiles.mapbox.com/mapbox-gl-js/v0.29.0/mapbox-gl.css' rel='stylesheet' />
    <style>
        body { margin:0; padding:0; }
        #map { position:absolute; top:0; bottom:0; width:100%; }
    </style>
</head>
<body>
    <div id='map'></div>
    <script>
        var map = new mapboxgl.Map({
            container: 'map',
            style: 'osm-bright-gl-style.json',
            center: [-122.46, 37.75],
            zoom: 11
        });
    </script>
</body>
</html>

In the next section, this index.html will be served with the CLI invoked npm node server http-server.

Connect the tile server with the display app and view the map in browser

Create a batch file to start up both the tile server and the app server

# kill any node processes left over from the last time this was run
sudo killall node
cd $MGJ
http-server -p 8888 --cors -o &
cd $MTS
node index.js

The -o argument to http-server opens a browser window automatically, so your SF map should be already visible.

Etc

The style file could have been served from the tile server instead of being physically placed in the same directory as the display app.

The app shown does depend on Mapbox GS JL code served by Mapbox servers, as can be seen from the lines :

<script src='https://api.tiles.mapbox.com/mapbox-gl-js/v0.29.0/mapbox-gl.js'></script>
<link href='https://api.tiles.mapbox.com/mapbox-gl-js/v0.29.0/mapbox-gl.css' rel='stylesheet' />

Therefore, the system shown in this post is not completely self-hosted. However the Mapbox GS JL code is declared as open source and the same code is supposed to be available on gituhub, so self-hosting should be possible.