How to load big geo-data file into web application

Question

I am building a web app where I show details in map. I load data using geojson file (190 mb).

I was looking in different possibilities to show the data, here there are:

• Leaflet GeoJSon: it should make it easy to load huge data, but I am not seeing effects on my work.

• Tile Layer http://bl.ocks.org/glenrobertson/6203331 It doesn't support some data viz libs that I am using

• Leaflet Vector layer http://geojason.info/2012/leaflet-vector-layers/ didn't get good results (having probs with data viz)

• TopoJson : resized the file from 190mb to 170mb. I still face the same problem.

The simplest way to load the file is using jquery "getJson".

jQuery.getJSON(url, function(data){
..
}

Someone suggested to stream the JSON data rather than bulk-load it. I have tried https://github.com/tmcw/geojson-stream . I couldn't get great results in all the cases.

Can you please suggest any better approach ?

Answer 1

It's obvious here that you can't expect to get 200 MB from a server to a client in a reasonable amount of time.

Your only options are to drastically simplify it (probably will resulting unusable information) or tiling it (either vector or raster tiling should work).

What you do depends on what you know and what you want to achieve. Here's what I suggest, in order of easiness:

1. use MapBox or CartoDB or equivalent to do the magic for you.
2. use GeoServer to do the backend stuff for you (you'll end up tiling with WMS and can use WFS for interactivity)
3. use TileJSON to tile up your vector data. This is more custom, if you're into that.

Answer 2

It really does depend on what you want the use to do with the web application. I asked a similar question here - Create interactive map with no server

1) If you want to just visually show the data then you could either load the data into a database (postgis, oracle etc) and put a geographic server (geoserver, mapserver, arcgis server) in front of that and deliver the data as a WMS to you choice of client (openlayers or leaflet). However this would mean running those applications in a production. The other way to do it is to pre generate the tiles using something like geowebcache and in openlayers or leaflet just read from a local Z/X/Y.png tile pyramid.

2) If you need the actual features (geometry) in the map application then you could either deliver the features to the client using WFS (you would still need database + server) or use UTF Grids which can be created in TileMill or a custom renderer.

Openlayers with UTFGrid information can be found here - http://dev.openlayers.org/docs/files/OpenLayers/Layer/UTFGrid-js.html

And the details on UTF Grids from MapBox can be found here https://www.mapbox.com/blog/how-interactivity-works-utfgrid/

UTF Grid writers here https://github.com/springmeyer/utfgrid-example-writers

I was trying with similar size data and TopoJson, GeoJson and neither worked for the amount of data you are talking about. So I used UTF Grids