Interesting Tech Projects
Archive for July, 2009
OSM and Tucson
Jul 28th
I noticed that various cities and counties/states have pages in the OpenStreetMap (OSM) wiki, but Tucson didn’t. I’ve now fixed that with a page for Tucson that provides a list of projects with percentage of completion. I hope that will encourage people to contribute by giving them ideas of where to start.
Personal Open Street Map Goals
Jul 25th
I’ve decided to write down my goals. These are long-term; if I can get them done in six months that would be great.
1. Large Scale Features
Adding national forests, parks and preserves, wilderness areas and the trails inside those areas for Arizona. I am perhaps 70% done with this goal. The benefit is vastly improved recreation maps for the six million people in the state. The area is huge (larger than the UK) and the total number of nodes is probably in the 100,000s.
2. Medium Scale Features
Adding key features in the city of Tucson. This includes major rivers and washes, large buildings such as hospitals, malls and big box stores, realigning primary and secondary roads. To date I have added at least one of each of these features. The benefit is a good overall map for the one million people who live in the Tucson area.
3. Small Scale Features
I plan to completely map in as much detail as possible the nine square miles or so around my home. This includes realigning all the streets, adding buildings which are not houses, adding recreation areas, ensuring all streets are named, etc. Of the three goals this is probably the most work. The number of people living in the area isn’t that much, but I intend this to be an example for future Tucson mappers.
Adding Rivers To Open Street Map
Jul 25th
Some rivers are small enough that they can be added with just a line. But how are wider rivers added? They are polygons that can stretch hundreds of miles. How do you stop and start editing or does the entire river somehow have to be added at once?
The answer is to use the tag waterway:riverbank and build the river up from polygons that are next to each other. This diagram shows:
Seems simple enough. But what about bridges. The bridge:yes page explained. Just cut up the road and mark the section on the bridge with bridge:yes. The layer option was a bit ambiguous so to be sure I added layer:1 for the bridge.
So I tried added a section of the Pantano Wash in Tucson. After waiting 15 minutes or so the result was added to the map:
Looking good. Now to add some more.
Open Street Map Tags
Jul 25th
Finding out which tags to use when can be difficult. JOSM includes a presets menu that helps, but what is the difference between landuse:retail and landuse:commercial?
I had an idea of the difference but before using these tags for the first time I needed confirmation. After some digging I found the Map Features page in the OSM wiki. It confirmed my thoughts. Retail = shops, commercial = offices.
Preparing to Map Summerhaven
Jul 20th
The village of Summerhaven is in the Santa Catalina mountains just north of Tucson. The US Tiger data imported into Open Street Map is a mess, and needs to be fixed. Unfortunately the satellite images from Yahoo are not much help for that place.
At the weekend we plan to undertake our first mapping trip. This will involve driving around all the roads in the village with the GPS unit turned on and taking pictures of street signs and points of interest.
Back home JOSM can be used to load in the track log and match up the timestamps of the pictures with where the GPS unit was at the time. Hopefully that will give me all the data I need to rip out Summerhaven and draw it from scratch, but in the correct place and with more detail.
Here is a picture of what Summerhaven current looks like.
Open Street Map
Jul 19th
Open Street Map (OSM) is a world map that anyone can edit, just like a wiki. I used the data in a few projects but seeing as the US road system was generated from a mass import of US Government data, I thought that it was largely complete and not much to do. This was, of course, wrong.
Back in May I was in East Yorkshire, England and I added a trail to the map. A local OSM mapper contacted me about it. When I got back to Tucson, Ariz. I was inspired to look more closely at the local data to see what was missing.
I found that most of the streets were not aligned correctly. Also the National Forests were missing, plus a nearby National Park. There is a lot to do in the US, that’s for sure.
I started off by realigning the streets in my area and completed four square miles. But what really interested me was the forests. A bonus is that US Government data is in the public domain and can be freely used for OSM.
I’ve now spent the last month, off and on, working on this. So far I have added all the National Forest boundaries in Arizona and New Mexico, some grasslands in New Mexico and Oklahoma and a Nature Preserve in New Mexico. I’ve also added Saguaro National Park East and the trails in the park. Currently I am working on adding the Wilderness Areas for Arizona and New Mexico and the forest trails.
I encourage you to take a look at the map in your area and see what you can do!
Getting US Government Shapefiles Into OpenStreetMap
Jul 10th
I recently embarked on the process of adding the shapefiles for Region 3 of the US Forest Service into OpenStreetMap (OSM). This post describes the method I used. First a bit of background.
OSM is a free street level map of the world. Just like Wikipedia is a encylopaedia that anyone can edit, OSM is a map that anyone can edit. The data is licensed under Creative Commons by attribution, which is a very liberal license.
In the US currently most of the data came from a mass import of the US Government’s TIGER data. This data has numerous problems, such as streets being in the wrong places and regions like forests are missing completely.
Fortunately some regions of the US Forest Service make good quality shapefiles available of the forest boundaries and I live in one of those regions – the Southwestern Region. It’s just a matter of converting the shapefiles into the correct format, adding some descriptions and uploading them to the OSM server.
There are several challenges with getting the data into OSM. Firstly the shapefiles are in the Lambert Conformal Conic projection, which is not what OSM uses. Secondly OSM doesn’t use Shape Files, it uses it’s own XML data format. Thirdly the data has to be “tagged” in the correct way for the forests to be rendered correctly. This becomes more complex if there are holes in the forest.
The first stage is to find the shapefiles and what projection they are in. In my case the US Forest Service provided the projection information with the data files. I then reprojected the data and converted to a OSM file format using the following steps:
- Open the shapefile in MapWindow GIS
- Use the GIS Tools menu to reproject the data to WGS84
- Save
- Open the reprojected data in GPS TrackMaker
- Save as GPX file
- Use GPSBabel to convert from GPX to OSM
gpsbabel -i gpx -f smallpiece.gpx -o osm -F smallpiece.osm
Currently the best editor to use for OSM is JOSM. This is a Java application that allows data to be added, edited, deleted and tagged. Don’t use the default memory allocation, instead allocate more using:
java -jar -Xmx256M josm-latest.jar
There are some essential plugins that are required. Install utilsplugin, validator and waydownloader. Here are the steps I used to edit and tag the data. There may be mistakes, so use at your own risk.
- Open in JOSM
- Simplify using Shift-Y
- Select all nodes in a single ring (closed loop) and choose Combine Way
- Select all outer ways
- Add way tags (see below)
- Select all inner ways and delete all tags
- Select all ways with no holes
- Create a relation
- Add the selected ways to the relation
- To the left of the way in the dialog window set the role to “outer” for all ways. Click on each way to see which it is on the map
- Add the tags for the relation (see below), setting the type to “boundary”
- Select each way with holes in turn, along with the holes
- Create a relation
- Add the selected ways to the relation
- To the left of the way in the dialog window set the role to “outer” for outside rings. Set the role to “inner” for inside rings.
- Add the tags for the relation (see below), setting the type to “multipolygon”
- Split large ways into smaller ways of 2,000 nodes or less
- Validate and fix errors. Ignore warnings about unclosed ways.
The tags I use for ways are:
- uuid: username_yyyymmddn
- attribution: US Forest Service
- boundary: national_park
- landuse: forest
- leisure: nature_reserve
- name: Coronado National Forest (example)
- wood: mixed
The tags I use for relations are:
- uuid: username_yyyynnddn
- name: Coronado National Forest (example)
- landuse: forest
- type: multipolygon (when there are holes)
- type: boundary (where there are no holes)
- boundary: national_park
Do not add tags to inner ways or the nodes. GPSBabel automatically adds tags, so delete those.
The uuid’s are a unique identifer that I can use to search for this particular set of data later, if needed. It is constructed from my OSM user name, the date and the upload number for the day. For example:
FooBar_200907104
would be the fourth upload for July 10th 2009 by the user FooBar. I use this scheme because I can work out the uuid’s from just an upload date and it is very unlikely that someone else would use the exact same uuids.
