Serving your map tiles 30 times faster

17 January 2012 — Categoria: Django , GIS , Python , Sysadmin , Tech , WebGis

Recently I’ve been experimenting with TileStache tiles caching system, which is similar to TileCache and other tile caching systems in scope.

TileStache revealed itself as a powerful and flexible product and was very good to cache tiles generated with my Django-backed mapscript-powered WMS server. Caching and serving tiles with TileStache was by far faster then running the whole mapscript WMS machinery.

But I wanted to do more: if I could just convince nginx to serve the tiles directly from disk, this would have been even faster. Of course for this to work, TileStache must be configured to cache tiles on disk, which is what you normally want.

nginx is really good at serving static files and it is luckily wery flexible and easy to configure (once you know what to do).

The following configuration uses named regular expression captures and a try_files instruction to check for the tile existance, if found, the tile is served directly, if not found the internal redirect to the @django named location takes place.
The 404 trick in the base location (/) was necessary to avoid duplication of the @django location content, but there might be a better way.

nginx configuration file

server {
    listen   88 default;
    server_name  localhost;
    access_log  /var/log/nginx/access.log;
    rewrite  "^/[a-z]{2}/(static|media)/(.*)$"  /$1/$2;
    root /www/django_app/;
 
    location  /media/ {
        alias /www/django/contrib/admin/media/;
    }
 
    location  /static/ {
        alias /www/django_app/site_media/;
    }
 
    location ~ ^/(?<lang_code>..)/resource/tiles/1\.0\.0/(?<tile_path>.*)$ {
        alias /tmp/stache/;
        add_header X-Static super;
        try_files $http_host/$lang_code/$tile_path @django;
    }
 
    location / {
        add_header X-Static hit;
        error_page 404 = @django;
        return 404;
    }
 
    location @django {
        add_header X-Static miss;
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $http_host;
    }
}

Results

niginx is listening on port 88 and the dev server (I’m using werkzeug in this case) is listening on port 8080.

ab -c 100 -n 1000 http://localhost:88/it/resource/tiles/1.0.0/track_4/0/0/0.png
 
Concurrency Level:      100
Time taken for tests:   0.084 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      7553744 bytes
HTML transferred:       7302912 bytes
Requests per second:    11902.92 [#/sec] (mean)
Time per request:       8.401 [ms] (mean)
Time per request:       0.084 [ms] (mean, across all concurrent requests)
Transfer rate:          87804.31 [Kbytes/sec] received

With TileStache only:

ab -c 100 -n 1000 http://localhost:8000/track_4/0/0/0.png
 
Concurrency Level:      100
Time taken for tests:   2.944 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Non-2xx responses:      1000
Total transferred:      145000 bytes
HTML transferred:       0 bytes
Requests per second:    339.63 [#/sec] (mean)
Time per request:       294.438 [ms] (mean)
Time per request:       2.944 [ms] (mean, across all concurrent requests)
Transfer rate:          48.09 [Kbytes/sec] received

Conclusions

Unsurprisingly, the solution with files served directly is several times faster: 35 in this test case.

In a production environment, using mod_wsgi or uwsgi, the fugures may vary, but I’m confident that the overall speed boost is guaranteed.

Posted by Alessandro Pasotti at 20.03 Nessun commento

Lascia un Commento