TileStache revealed itself as a powerful and flexible product and was very good to cache tiles generated with my Django-backed mapscript-powered WMS server. Caching and serving tiles with TileStache was by far faster then running the whole mapscript WMS machinery.

But I wanted to do more: if I could just convince nginx to serve the tiles directly from disk, this would have been even faster. Of course for this to work, TileStache must be configured to cache tiles on disk, which is what you normally want.

nginx is really good at serving static files and it is luckily wery flexible and easy to configure (once you know what to do).

The following configuration uses named regular expression captures and a try_files instruction to check for the tile existance, if found, the tile is served directly, if not found the internal redirect to the @django named location takes place.
The 404 trick in the base location (/) was necessary to avoid duplication of the @django location content, but there might be a better way.

nginx configuration file

server {
    listen   88 default;
    server_name  localhost;
    access_log  /var/log/nginx/access.log;
    rewrite  "^/[a-z]{2}/(static|media)/(.*)$"  /$1/$2;
    root /www/django_app/;
 
    location  /media/ {
        alias /www/django/contrib/admin/media/;
    }
 
    location  /static/ {
        alias /www/django_app/site_media/;
    }
 
    location ~ ^/(?<lang_code>..)/resource/tiles/1\.0\.0/(?<tile_path>.*)$ {
        alias /tmp/stache/;
        add_header X-Static super;
        try_files $http_host/$lang_code/$tile_path @django;
    }
 
    location / {
        add_header X-Static hit;
        error_page 404 = @django;
        return 404;
    }
 
    location @django {
        add_header X-Static miss;
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $http_host;
    }
}

Results

niginx is listening on port 88 and the dev server (I’m using werkzeug in this case) is listening on port 8080.

ab -c 100 -n 1000 http://localhost:88/it/resource/tiles/1.0.0/track_4/0/0/0.png
 
Concurrency Level:      100
Time taken for tests:   0.084 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      7553744 bytes
HTML transferred:       7302912 bytes
Requests per second:    11902.92 [#/sec] (mean)
Time per request:       8.401 [ms] (mean)
Time per request:       0.084 [ms] (mean, across all concurrent requests)
Transfer rate:          87804.31 [Kbytes/sec] received

With TileStache only:

ab -c 100 -n 1000 http://localhost:8000/track_4/0/0/0.png
 
Concurrency Level:      100
Time taken for tests:   2.944 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Non-2xx responses:      1000
Total transferred:      145000 bytes
HTML transferred:       0 bytes
Requests per second:    339.63 [#/sec] (mean)
Time per request:       294.438 [ms] (mean)
Time per request:       2.944 [ms] (mean, across all concurrent requests)
Transfer rate:          48.09 [Kbytes/sec] received

Conclusions

Unsurprisingly, the solution with files served directly is several times faster: 35 in this test case.

In a production environment, using mod_wsgi or uwsgi, the fugures may vary, but I’m confident that the overall speed boost is guaranteed.

• part 1: Polymaps, TileStache and SVG CSS styling issues
• part 2: TileStache Django tiny wrapper

during my experiments I wanted to reach the goal of minimize both the bandwith consuption and the number of vector coordinates that the browser needs to handle, this means that if we need to show complex geometries, for example a set of mountain hiking paths for an overall of 250000 point, we absolutely need a way to simplify features at lower zoom levels.

TileStache doesn’t support this out-of-the box, so I started playing with a clone of the Vector provider and ended up with a vector-simplify branch in my fork.

What I’ve done was basically to add a new “simplify” provider configuration parameter, where you can enter the tolerance for Douglas-Peucker simplification algorhitm in pixel unit. The paramer can be set in TileStache config file as shown here:

  "layers": {
    "pmcollection":
    {
        "provider":
        {
            "name": "vector",
            "driver": "PostgreSQL",
            "parameters": {
                "dbname": "polymaps_test",
                "user": "xxx",
                "port" : "5433",                
                "table": "pm_points_pmcollection"
            },
            "verbose" : "true",
            "simplify" : 0.5
        },
        "allowed origin" : "*"
    }
 }

A picture is worth a thousand words

Some pictures might be the best way to show how this parameter affects the generated geometries:

This is a single tile, the original geometry is in red, the simplified geometry in green/yellow and an over-simplified geometry in blue.

The following overall views show all the details:

Original geometry

Over-simplified with tolerance 50

Simplified with tolerance 0.5

Open issues

The biggest issue I’ve encountered was about getting the tolerance in lat-lon WGS84 starting from pixel units, but let me explain why I choose pixel units in the first place: I think that pixel units are are the best choice because normally we don’t care about feature details which are below 1 pixel, we simply cannot see them. Using pixel units we also get the advantage of using a single number for all zoom levels, a more complicated approach would consist in using separate values for each zoom level. Translating pixel units to decimal degrees is not possible without approximations, I just hope that my approach works in most situations.

Another strange thing I noted in the library is that the bounding box used for clipping (which is a rectangle) is built using 16 points instead of 4. This means that after clipping we end up with a geometry that have many more points that it needs. I’m still waiting for a clarification I’ve asked in the TileStache ML, but in the meantime I patched my branch to use a normal 4 points rectangle and everything seems to work fine (and with smaller geojson responses).

Conclusions

TileStache confirmed to be a valuable tool for web GIS developers, it’s well written and supported and easily expandable, absolutely worths a try.

First, I would like to thank Michal for the quick patch inclusion, the Polymaps patch is still waiting for the pull and the Django admin patch is awating for somebody to take care of that 18-months-old bug.

My experiments have continued with Django integration, I just wanted to see how easy it was (and it was really easy indeed). There aren’t any clear advantages using a Django app instead of the standalone TileStache server, but writing the wrapper was a nice exercise to learn something more about TileStache and Polymaps, and having all the pieces into one place is always tempting for me (not to mention that you can dribble the CORS completely).

The wapper consists in a single router rule in urlconf:

    url(r'^tiles/(?P<layer_name>[^/]+)/(?P<z>[^/]+)/(?P<x>[^/]+)/(?P<y>[^/]+)\.(?P<extension>.+)$', 'pm_points.views.tiles', name='tiles_url'),

… a view for the tiles (get_config implements a singleton to avoid parsing TileStache config every time):

 
# proper imports go here...
 
def tiles(request, layer_name, z, x, y, extension):
    """
    Proxy to tilestache
    {X} - coordinate column.
    {Y} - coordinate row.
    {B} - bounding box.
    {Z} - zoom level.
    {S} - host.
    """
    config = get_config()
    path_info = "%s/%s/%s/%s.%s" % (layer_name, z, x, y, extension)
    coord, extension = TileStache.splitPathInfo(path_info)[1:]
    mimetype, content = TileStache.getTile(config.layers[layer_name], coord, extension)
    return HttpResponse(content, mimetype=mimetype)

… and a simple view for the map:

 
def home(request):
    """
    Pass layers by
    """
    config = get_config()
    layer_urls = []
    for l in config.layers:
        if isinstance(config.layers[l].provider, TileStache.Providers.Vector.Provider):
            layer_urls.append(reverse('tiles_url', args=[l, '{Z}', '{X}', '{Y}', 'geojson']).replace('%7B', '{').replace('%7D', '}'))
    return render_to_response('home.html', { 'layer_urls' :  layer_urls}, context_instance=RequestContext(request))

… and a template for Polymaps (I paste the js part only):

 
    var po = org.polymaps,
        map,
        layers = [],
        stylist;
        jQuery(function(){
            map = po.map()
                .container(document.getElementById("map").appendChild(po.svg("svg")))
                .center({lat: 40, lon: 0})
                .zoomRange([1, 16])
                .zoom(2)
                .add(po.interact())
                .add(po.hash())
                ;
 
            map.add(po.image()
                .url("http://s3.amazonaws.com/com.modestmaps.bluemarble/{Z}-r{Y}-c{X}.jpg"));
 
            stylist = po.stylist()
                .attr('class', function(d){
                    return d.geometry.type + '_' + d.properties.id;
                })
                .title(function(d){return d.properties.name;})
                ;
 
            {% for lay_url in layer_urls %}
            var layer = po.geoJson()
                .url("{{ lay_url|safe }}")
                .zoom(function(z){
                    return z;
                })
                .id('pmcollection')
                .on("load",  stylist)
                .on('show', stylist)
                ;
            map.add(layer);
            layers.push(layer);
            {% endfor %}
 
            map.add(po.compass()
                .pan("none"));
            map.container().setAttribute("class", "mymap");
        });

Quick and dirty, it also dynamically adds layers to the Polymap.

This story continues with: TileStache Vector simplify provider

The final goal of this experiments was the integration with Django.

Polymaps

Polymaps is a Javascript library aimed to display svg maps in a web page, recently I’ve watched to an interesting talk about a Django-powered app built on top of Polymaps:

The main reason to give Polymaps a try is the interesting support to tiled vector layers, as far as I know a unique feature.

I’ve found the documentation of Polymaps a bit obscure and still incomplete (I just hope it isn’t another facade Open Source product where the lack of documentation and community support limits the use to commercial projects).

A quick look in the code shows no comments and rather obscure variable names, debbugging this library would certainly be a nightmare if you’re not the author, if you compare the source code with OpenLayers’s code you get the idea.

BTW I managed to make it work in a few hours (with all the server stuff).

TileStache

TileStache is a python library aimed to build, cache and manage map tiles. It’s similar to TileCache in scope and uses the well known Mapnik renderer while the data provider is implemented with shapely and OGR and can read the most widely used formats (postgis, shapefile, geojson).

TileStache is the ideal companion to Polymaps because it can handle and cache vector tiles (geojson) through the Vector class (don’t even try to use the PostgreSQL class you can find in Goodies, it’s kind of unofficially deprecated and it will let you waste your time).

I had to fork the project on github and made a pull request to fix the missing “port” parameter to the DB connection since I’m using a not-standard port.

SVG CSS

I admit I am a true beginner with SVG and while I was trying to experiment with the stylist features of Polymaps  I discovered some counter-intuitive (at least for me, coming from XHTML CSS) behaviours:

You can define styles in three ways:

1. CSS rules
2. “style” attribute of the SVG element
3. individual attributes (“stroke”, “fill” etc. etc.) of the SVG element

Strange enough, the priority is top-down, while I would have expected that point 3 had a higher priority.

Another thing worths noting is that what distinguish a polygon from a linestring in the Polymaps generated “path” SVG element is just that the “d” element (which contains the geometry) ends with a “Z”.

This is a linestring:

<path d="M101.99999715555555,189.9999971369224
L75.00000142222234,219.00000190473258
L75.00000142222234,242.49999834108655
L94.49999928888906,242.99999806558844
L102.49999928888906,255.4999976705726
L101.99970702222231,256"/>;

This is a polygon:

<path d="M95.07080533333351,256.00000052462633
L0,196.67516186124612
L0,192.0000022835775
L0,128.0000001888236L0,70.92093168072711
L175.92470186666674,0.000006325172648757871
L192,0.000006325172648757871
L256,0.000006325172648757871
L256,64.00000444302805
L256,128.0000001888236
L256,160.99230434784874
L113.66593991111131,256.00000052462633Z"/>;

Styling the plygon fill without hitting linestring in a GEOMETRYCOLLECTION layer turned out to be quite troublesome.

I ended up with a CSS-3 attribute CSS rule:

.layer path {
fill: cyan;
fill-opacity: .85;
stroke: #012;
stroke-linecap: round;
}
 
.layer circle {
fill: cyan;
fill-opacity: .85;
stroke: #012;
}
 
/* Don't fill in SVG */
.layer path:not([d$="Z"]){
fill: none;
stroke-width: 3px;
}

I wonder if there is a better way…

This story continues with: TileStache Django tiny wrapper.

GeoContent Plus Logo

  native!

Browse GeoContent Plus documentation!

Try GeoContent Plus demo!

If you are looking for the old GeoContent version for Joomla! 1.5, please check the original announce.

Introduction

The Map

GeoContent Plus is a Joomla! 1.7.x and 2.5.x component that let you easily embed beautiful Google Maps or OpenLayers Maps into your Joomla! 1.7.x and 2.5.x website.

Standard Joomla! articles, can be bound to geographic features such as POINTs, LINEs and POLYGONs and the resulting maps can be shown in the article page and in one or more overview maps.

A single GeoContent Item can be represented by more than one geographic type: so that you can have simultaneously a set of POINTS, some POLYGONS and some LINES for a single Joomla! article.

GeoContent Items on the map are grouped by GeoContent Layers, graphic appearance of GeoContent Items can be fully customized defining styles attributes in the GeoContent Layers configuration panel.

GeoContent Plus was created as a sort of Geographic Index to Joomla! articles, the earliest versions had a tight binding between Joomla! articles and the geometric features that GeoContent Plus was going to show on its maps. Suggestion from the users made GeoContent Plus subsequently evolve to cover more general use cases, it is now possible to createGeoContent Items which are completely unbound from existing Joomla! articles.

The Front-End editor window

The new Access Control List (ACL) features introduced by Joomla! 1.6 are now fully supported by GeoContent Plus allowing a fine grained control over the front-end and back-end user access to the different operations available in the software.

GeoContent Items are also language-aware: they can optionally make use of the same Joomla! language support system recently introduced for Joomla! core articles.

Download

ComGeocontent is open source software, distributed under AGPL software license and it is also free software, where “free” means that you can do (almost) whatever you want with this software but it does not mean that it costs nothing. We distribute the component only in bundle with paid 12 month support service that costs 45 €.

To receive a copy of GeoContent you are kindly requested to pay us 45 €, using the buttons located at the top or at the bottom of this page. You will receive a regular invoice at the end of the month. After a successful payment,  an email with the download link will be automatically sent to you PayPal’s account email address.

The money you pay will be used to further develop GeoContent and to provide professional assistance and support to those that will need it.

Together with 12 months support you will get lifetime software updates, that means that you can pay just once and use the software forever in how many Joomla installations you like.

Please use the PayPal link at the top of this page, specifying the email where the component will be sent.

Thank you for supporting GeoContent project!

Key features

QGIS logo

Today I found some time to test some new amazing QGIS features. One of the most interesting additions is the WMS integrated server.

I usually test all new projects in my user directory, I created a folder ‘~/public_html/qgis-client/’ to host this test.

Next step was to enable CGI handling, since I run apache under Linux and all I had to do was to create a file named ‘.htaccess’ in the ‘~/public_html’ folder:

AddHandler cgi-script .fcgi .cgi
Options +ExecCGI
SetEnv LD_LIBRARY_PATH /home/myuser/app/lib

This few lines, enable CGI and add a search path to the library loader, this was necessary for the handler to load my QGIS trunk build, located in ‘~/app/’.

Now I needed to symlink the qgis-mapserv.fcgi executable into the project folder:

cd ~/public_html/qgis-client/
ln -s ~/app/bin/qgis_mapserv.fcgi .
~/app/bin/wms_metadata.xml .
vim ~/app/bin/wms_metadata.xml

The last command allows the customization of the GetCapabilities response.

I was now able to serve a qgis project with the WMS protocol, the project location can be specified with the map parameter, like this:

http://localhost/~myuser/qgis-client/qgis_mapserv.fcgi?map=/home/ale/public_html/qgis-client/qgis-client.qgs

GFOSS is the Italian chapter of OSGEO fundation.

I am a proud member from its birth and I participate to almost all events.

About GFOSS Day:

GFOSS day is an yearly meeting where all the people interested in geographic open source free sotware can meet to share ideas and experiences.

This year 2010  the event will last two days (18 and 19 november) with conferences, tutorials and workshops.

Abbiamo partecipato direttamente come associazione GFOSS per l’informatica geografica libera all’incontro sui sistemi informativi territoriali con una presentazione sulle applicazioni di web mapping e sull’interoperabilità dei servizi webgis.

Potete scaricare le slide del mio intervento a questo indirizzo:

Slide linux day 2007

Interessantissime novità anche dal punto di vista della certificazione, da poco, grazie al lavoro di Marina Cabrini e Andrea Giacomelli nonché di diversi membri della comunità GFOSS, è possibile ottenere al certificazione GIS ECDL riconosciuta a livello europeo utilizzando solo software libero. Un altro importante traguardo è stato raggiunto!

Maurizio Napolitano ha presentato l’esperienza Web Valley , una iniziativa decisamente interessante di didattica basata sull’informazione geografica e sul software libero.

Un sincero ringraziamento a tutti i relatori, gli organizzatori e i partecipanti di questa bella giornata.

Arrivederci al Linux Day 2008!

The advantages of such a system are those of typical webservice:

• fully decoupled from the GUI, the GUI could be written in any language that talks REST
• plain text mapfiles as the storage system

The service is being written in PHP and the RESTFUL part is managed with PHP Tonic library.

A few examples of typical URLs:

• GET http://www.mymaps.com/maps – retrieve a list of mapfiles
• POST http://www.mymaps.com/maps – creates a new mapfile with the JSON data passed on post
• GET http://www.mymaps.com/maps/my_first_map.json – retrieve the map data as JSON
• GET http://www.mymaps.com/maps/my_first_map.xml – retrieve the map data as XML
• GET http://www.mymaps.com/maps/my_first_map.map – retrieve the map data as plain text mapfile
• PUT http://www.mymaps.com/maps/my_first_map.json – alter the map with JSON data
• PUT http://www.mymaps.com/maps/my_first_map.xml – alter the map with XML data
• GET http://www.mymaps.com/maps/my_first_map.xml/layers – retrieve the map layers as XML
• GET http://www.mymaps.com/maps/my_first_map.xml/layers/my_first_layer – retrieve the map layer data as XML