The Mythical City of Atlantis – just as legendary as GeoTrellis 0.8′s advanced geoprocessing power! (image credit)

The GeoTrellis team is very excited to announce the availability of GeoTrellis 0.8 (codename “Atlantis”), which is a major new release that is a huge step forward towards our goal of a general purpose, high performance geoprocessing library and runtime designed to perform and scale for the web.

First of all, you should check out our new documentation site — geotrellis.github.com/ — which features a great deal of new documentation and tutorial material. Credit is due to Rob Emanuele for the terrific new organization and layout.

Secondly, take a look at the Azavea Labs blog Adam Hinz recently published (example running above) which describes process of building a kernel density service using GeoTrellis that features some of the new 0.8 functionality. Adam’s post details step-by-step the development of a WMS service that should be helpful as you think about how you might develop your own services with GeoTrellis. It is a companion blog to the Azavea Atlas blog series being written by John Branigan that explores map algebra concepts through the process of creating a site suitability model using wildlife habitat preferences.

As you delve into GeoTrellis 0.8 in more depth, here are some new features you may want to explore:

• A new suite of raster operations in the focal category of map algebra.
  • For more information, see: geotrellis.github.com/operations/raster/focal.html
• Simplified raster rendering with a built-in suite of color ramps for data cartography.
  • For more information, see: geotrellis.github.com/overviews/rendering.html
• Zonal summary (by feature) operations, including a caching mechanism for tiled rasters for improved performance.
  • For more information, see: geotrellis.github.com/operations/raster/zonal.html
• New vector feature framework for operations on points, lines, polygons with a suite of vector operations, as well as re-engineered rasterization operations.
  • For more information, see: geotrellis.github.com/overviews/vector.html and geotrellis.github.com/operations/feature.html

Please let the team know — via the #geotrellis channel on Freenode IRC or the geotrellis-user Google Group mailing list — if you have any comments or suggestions. We will likely release a minor bugfix release (0.8.1) in the future.

Github: github.com/geotrellis/geotrellis
Maven repository: https://oss.sonatype.org/content/repositories/releases/
API Scaladocs: geotrellis.github.com/geotrellis/latest/api
Issue tracker: https://github.com/geotrellis/geotrellis/issues?milestone=3&state=open
Mailing list: https://groups.google.com/group/geotrellis-user
IRC: #geotrellis on freenode

GeoTrellis is released under the GPL V3 license.

To go along with the recent Azavea Atlas post, we’re going to build a WMS service for the Western Jackalope using GeoTrellis.

GeoTrellis is a high performance geoprocessing engine and programming toolkit developed here at Azavea.

There’s an example showing the Western Jackalope sightings running at 207.245.89.238/labs1-demo/index.html.

Scala Environment

We’re using the same made-up dataset that John used in the Atlas Article. The first step is getting our environment setup. If performance is key, we’ve found the Oracle JVM to be a bit faster than OpenJDK. For simplicity, we’re going to use the OpenJDK found in the standard apt repo:

To build scala projects we use “sbt”. If you don’t have it, you can install it:

To get moving we need two boilerplate files. The first is our project definition (which lives in the root in build.sbt)

The second file is needed to explain to GeoTrellis what classes we want exposed and on what port by updating the settings in the application.conf file

To make sure everything is looking good we’re going to setup a simple hello world service. We’re using Jersey/JAXRS for web services in this tutorial but you should feel free to use whatever.

Testing Service

At this point we can do “./sbt run” and see our service startup by going to localhost:8888/hello and you should see “Hello World”. Jersey uses annotation to determine what methods to expose. We use the “@Path(…)” annotation to represent that path of a server (“/hello” in this example). Decorating our method with @GET exposes it via GET requests.

Formatting Our Data

Before we can analyze the data we need to grab some data and prep our environment. We use GeoTools to parse our shapefile in to GeoTrellis features.

The last few lines set up our GeoTrellis server executor. Usually GeoTrellis reads a catalog file to determine where relevant raster data is on the server. Since we’re only generating rasters (not reading them) we used a blank catalog. The server is responsible for running operations.

The last few lines setup our color handling. All of the regular color ramps have their alpha values set to 100%. For our heat map we want low values to fade away. In the above case the lowest values are dropped completely (0×00000000) and the remaining values are set to 50% opacity.

Kernel Density

Here’s a kernel density service that we’ll be using:

We start out grabbing our bounding box (xmin, ymin, xmax, ymax), the width and height, size of our kernel and the kernel’s spread (standard deviation in meters). “Context” is a global object in “demo.azavea” that we use to store some static stuff, like our feature and server.

GeoTrellis provides a few operations for converting strings to relevant objects:

• ParseRasterExtent
• ParseInt
• ParseDouble

More can be found in the “geotrellis.rest.op.string” package

A common kernel to use for Kernel Density operations is the Gaussian, which we create using CreateGaussianRaster. In our example we allow the standard deviation to be set by the service and set the amplitude to 100.

The main show is the KernelDensity operation. We’re using the feature set that we loaded from the shapefile. The second argument is a function to convert our feature value in a number. In our case each feature is a Point[Int] so we use the identity function.

This kernel density server represents everything in meters and kilometers so we need to scale the values appropriately before we dive in to the rendering (spreadOp and sizeOp).

The last 10 lines render the PNG and and return it. Since we’re generating tiles on the fly we’re not going to use quantile breaks because the tile edges won’t line up. Instead, we create our own break array and use RenderPng directly.

Resources

For more information checkout geotrellis.github.com/ or chat with us at #geotrellis on freenode.

The code can be found on github at https://github.com/geotrellis/labs1-demo

In case you haven’t heard, Azavea likes to build beautiful and functional web applications. Particularly, geospatial applications.  Currently, each app involves a great deal of JavaScript for user interactions and dynamic content. This has presented a unique set of challenges when it came to code quality tools and best practices with respect to unit testing and Test Driven Development (TDD). I’ll address JavaScript unit testing in .NET in this post, since many of our projects use these two technologies.

Unit Testing / TDD

As it turns out, there aren’t a whole lot of great tools in the .NET ecosystem that will unit test JavaScript source files.  Sure, you could install nodejs on a windows box, or even run your tests in the browser through QUnit.  It really comes down to a matter of taste, and my preference is to have fewer moving pieces in the build chain if possible.  It turns out that JSTest.NET fits into our .NET ecosystem nicely, so I wanted to give it a shot (it also helps that it’s a package available in NuGet).

One of the interesting parts about JSTest.NET is that it runs in the Windows Scripting Host, which is a neat way to run JavaScript programs from the command line in Windows. If you’ve ever double-clicked on a JavaScript file, and received a cryptic error, you probably ran the file through the Windows Scripting Host without realizing it. Since the executing process is this scripting host, you don’t have access to normal browser stuff, like “window” or “document”. This turns out to be important, especially in web applications.

» Continue Reading

Implemented Using a Google Maps Android API v2, TileLayer

Intro

This brief article documents some early exploration into using WMS (Web Map Service) with the new Google Maps V2 API. It is intended as a reference to help someone trying to get WMS tiles (IE from GeoServer) onto an Android map.

WMS is used to serve map tiles over HTTP by back end frameworks like GeoServer.  Some set of geo-referenced data, typically shape files or data stored in a PostGIS database, are returned as raster map tiles.  In the past, this data has been consumed by web applications using a client library such as Leaflet or OpenLayers.  With Google’s v2 mapping API for android, it is now relatively straightforward to build Android apps that combine WMS tiles with Google’s base maps and other data such as vector shapes and map markers.

For basic getting started info for v2 Maps, see the Google Developer’s site for the v2 API. This article assumes a working v2 setup with the sample code running without error. After downloading the google play SDK and setting up the library make sure you can view the TileOverlayDemo ($ANDROID_SDK_ROOT/extras/google/google_play_services/samples/maps)

PhillyTreeMap Android App (Proj. Release Spring 2013) showing WMS technique described in the article.

 Extending the UrlTileProvider class.

The v2 API provides the UrlTileProvider class, a partial implementation of the TileProvider class which allows developers to pull in map tiles by composing a URL string.

The API to UrlTileProvider is its getTileUrl method. To request a WMS tile,
we override this method to compose the right URL, and the Android mapping SDK does the rest for us. It seems simple enough, but the problem is that the signature of getTileUrl which is getTileUrl(int x, int y, int zoom) provides tile indexes (x and y) and a zoom level, but WMS requires that we provide a bounding box (xmin, ymin, xmax, ymax) in the request URL. The x, y, zoom parameters provide us enough information to figure this out, but we have to do a little bit of math.

Calculating the map bounds

We know the bounds of the entire map which is square. (roughly -20037508m to 20037508m in both directions using Web Mercator. See the graphic below or the map tiler site for exact values.) We don’t use Latitude/Longitude though, because it is unprojected, and this will cause map distortions.

From the google api docs for TileOverlay:

Note that the world is projected using the Mercator projection (see Wikipedia) with the left (west) side of the map corresponding to -180 degrees of longitude and the right (east) side of the map corresponding to 180 degrees of longitude. To make the map square, the top (north) side of the map corresponds to 85.0511 degrees of latitude and the bottom (south) side of the map corresponds to -85.0511 degrees of latitude. Areas outside this latitude range are not rendered.

Dividing by the number of tiles for a given zoom level.

The number of tiles in either x or z at any zoom level is n = 2^z. With this, and the bounds of the map, we can figure out the size of the tile. Using this information combined with the maps origin (see graphic) and the x, y, zoom data for a given tile, we can find out its bounding box.

Again from the google api docs for TileOverlay:

At each zoom level, the map is divided into tiles and only the tiles that overlap the screen are downloaded and rendered. Each tile is square and the map is divided into tiles as follows:

• At zoom level 0, one tile represents the entire world. The coordinates of that tile are (x, y) = (0, 0).
• At zoom level 1, the world is divided into 4 tiles arranged in a 2 x 2 grid.
• …
• At zoom level N, the world is divided into 4N tiles arranged in a 2^N x 2^N grid.”

Summary

• zoom level: z = [0..21]
  (See GoogleMap.get[Min|Max]ZoomLevel)
• map size: S = 20037508.34789244 * 2
  This constant comes from converting the lat/long values above to EPSG:900913, Web Mercator. Again, see this page on maptiler for a fantastic visual explanation.
• tile size = S / Math.pow(2, z)
  So @ zoom level 0, S is the full map, at zoom level 1 there are 2×2 tiles, and at zoom 3 there are 8×8 tiles and so forth.
• tile origin = (-20037508.34789244, 20037508.34789244)
• minX of the tiles bbox (for tile index x,y) = origin.x + x * S
  Where x is the tile index in the east-west direction passed to the getTileUrl function discussed above.
• maxX of the tiles bbox (for tile index x,y) = origin.x + (x+1) * S
  x+1 because we are looking for the right edge of the tile.
• minY of the tiles bbox (for tile index x,y) = origin.y + y * S
• maxY of the tiles bbox (for tile index x,y)= origin.y + (y+1) * S

Demo Code

In addition to the following code snippets, I’ve put a sample project on github.

Here is a WMSTileProvider class, which inherits from UrlTileProvider. It supports the above bounding box calculation.

import com.google.android.gms.maps.model.UrlTileProvider;

public abstract class WMSTileProvider extends UrlTileProvider {

    // Web Mercator n/w corner of the map.
    private static final double[] TILE_ORIGIN = {-20037508.34789244, 20037508.34789244};
    //array indexes for that data
    private static final int ORIG_X = 0; 
    private static final int ORIG_Y = 1; // "

    // Size of square world map in meters, using WebMerc projection.
    private static final double MAP_SIZE = 20037508.34789244 * 2;

    // array indexes for array to hold bounding boxes.
    protected static final int MINX = 0;
    protected static final int MAXX = 1;
    protected static final int MINY = 2;
    protected static final int MAXY = 3;

    // Construct with tile size in pixels, normally 256, see parent class.
    public WMSTileProvider(int x, int y) {
    	super(x, y);
    }

    // Return a web Mercator bounding box given tile x/y indexes and a zoom
    // level.
    protected double[] getBoundingBox(int x, int y, int zoom) {
    	double tileSize = MAP_SIZE / Math.pow(2, zoom);
    	double minx = TILE_ORIGIN[ORIG_X] + x * tileSize;
    	double maxx = TILE_ORIGIN[ORIG_X] + (x+1) * tileSize;
    	double miny = TILE_ORIGIN[ORIG_Y] - (y+1) * tileSize;
    	double maxy = TILE_ORIGIN[ORIG_Y] - y * tileSize;

    	double[] bbox = new double[4];
    	bbox[MINX] = minx;
    	bbox[MINY] = miny;
    	bbox[MAXX] = maxx;
    	bbox[MAXY] = maxy;

    	return bbox;
    }

}

You might use this class in a factory class as follows:

import java.net.MalformedURLException;
import java.net.URL;
import java.util.Locale;
import android.util.Log;
import com.google.android.gms.maps.model.TileProvider;

public class TileProviderFactory {

	private static final String GEOSERVER_FORMAT =
    		"yourApp.org/geoserver/wms" +
    		"?service=WMS" +
    		"&version=1.1.1" +  			
    		"&request=GetMap" +
    		"&layers=yourLayer" +
    		"&bbox=%f,%f,%f,%f" +
    		"&" +
    		"&" +
    		"&srs=EPSG:900913" +
    		"&format=image/png" +				
    		"&transparent=true";	

	// return a geoserver wms tile layer
	private static TileProvider getTileProvider() {
		TileProvider tileProvider = new WMSTileProvider(256,256) {

	        @Override
	        public synchronized URL getTileUrl(int x, int y, int zoom) {
	        	double[] bbox = getBoundingBox(x, y, zoom);
	            String s = String.format(Locale.US, GEOSERVER_FORMAT, bbox[MINX], 
	            		bbox[MINY], bbox[MAXX], bbox[MAXY]);
	            URL url = null;
	            try {
	                url = new URL(s);
	            } catch (MalformedURLException e) {
	                throw new AssertionError(e);
	            }
	            return url;
	        }
		};
		return tileProvider;
	}

}

So your Activity code (again see the sample Google maps code referenced above) would have the following calls to add the overlay:

public class MapDisplay extends android.support.v4.app.FragmentActivity {

	//...

	private void setUpMap() {
	    TileProvider tileProvider = TileProviderFactory.getTileProvider();
	    mMap.addTileOverlay(new TileOverlayOptions().tileProvider(tileProvider));    
	}

Conclusion

This article presented a demonstration of a simple WMS client using Google’s Android v2 Maps API. It covered the math involved with converting from tile index/zoom level to Web Mercator bounding box, and showed how to compose a URL using these values and an instance of Google’s UrlTileProvider class.

We are proud of the recent launch of the Chicago Early Learning portal. Its launch was well covered, and the portal has been well received.  The development of the site was in collaboration with the SmartChicago Collaborative, a start-up that formed as a partnership between the City of Chicago, the John D. and Catherine T. MacArthur Foundation, and The Chicago Community Trust.

The Early Learning portal has information about early childhood learning centers in and around Chicago.  It used to be difficult for parents and families to collect, explore, and share this information. Now that all the early learning centers in the city are mapped and listed in one place, it’s easier for parents and families to discover what services are available.

The portal addresses a simple data problem once the information has been collected. There is very little geographic processing that occurs in the application, and the portal’s main function is  to display location data.  It’s not that often that a school will move, so most of the data is relatively static.

Working on a web application with only a small amount of geographic processing was a little unusual for us. While unusual, we were pretty excited about it because we focused heavily on the user experience of the portal, in order to make it easier to use.  We chose technologies that were open and made the portal usable on desktops, tablets, and smartphones. The Google Maps API (which includes Geocoding and Directions), along with Twitter Bootstrap and Twilio enables the application to play friendly with many different devices and capabilities.

Not only that, but we’re excited about what’s coming next with the City of Chicago. The SmartChicago Collaborative is also working with the City of Chicago on Windy Grid, a massive, real-time data infrastructure. Building that infrastructure will enable much deeper analysis and hopefully connect many different agencies and their data tools together.

Deep analytics and big data at the city level would be really exciting, and we’re really happy to see the City of Chicago embracing these goals.  It means that web applications like the Chicago Early Learning portal are just the tip of the iceberg, and we’re anticipating great things in the future, both from the SmartChicago Collaborative, and the City of Chicago.