Embedding a custom Google mashup
Problem: You need to display GIS data on a webpage on top of a familiar basemap with basic zoom and pan tools.
There are many ways to do this. The one I will explain today doesn’t require programming skills, specially configured servers or proprietary software. Note: I borrowed heavily from Havard University’s Google Maps tutorial for this entry. They have an excellent free web-based GIS Technical Training Workshop Series.
The basic map they provide is all you really need to get started. The HTML file is structured and commented in such a user-friendly way that you will be able to start editing it to make common customizations to your map almost immediately. No need to wade through the Google Maps API Documentation until you’re ready to try something more complicated.
Go to http://maps.cga.harvard.edu/samples/basic.html and save the html source document (basic.html) to your harddrive. Then open it in a text editor.
The first thing you will probably want to change is the area of the world that the map is showing. To do that, change this bit of code, which is right now centered and zoomed to show the United States.
//Sets the initial map location and zoom level (latitude, longitude) in decimal degree format (zoom level) 1 - 21
gmap.setCenter(new GLatLng(39.10960, -96.5), 4);
For my example, I’m going to make a map of the recent Haiti earthquake. There’s been a lot of that going on recently as geo-Pros have been banding together to speed information to the relief workers.
I’ve changed my center coordinates to the location of the main quake, and upped my zoom so Hispaniola island fills the screen:
gmap.setCenter(new GLatLng(18.44, -72.54), 7);
Now, I’m going to change the starting style to make it a little less busy, since I’m going to be adding things. Uncomment the second line in the code here.
//Sets the initial map to load. Change between PHYSICAL, SATELLITE, HYBRID, NORMAL. Default is NORMAL
//gmap.setMapType(G_PHYSICAL_MAP);
When commented out, it will default to NORMAL. If you want to change that, you will need to delete the // in front and change it to one of the options above. Like so:
gmap.setMapType(G_HYBRID_MAP);
Note: PHYSICAL is an attractive terrain map that is not available in standard google maps. It’s tempting to use it, but it’s still too bright for the things I’m going to add next to show up well.
My map is centered on the location of the quake, but I also want to add a pointer to show exactly where it happened. To do that, insert the following code after the line where you changed the background map style.
//Adds a marker with window information
function createMarker(point,html) {
var marker = new GMarker(point);
GEvent.addListener(marker, "click", function() {
marker.openInfoWindowHtml(html);
});
return marker;
}
var point = new GLatLng(18.44,-72.54);
var marker = createMarker(point,'Magnitude 7.0 Earthquake Jan 12, 2010');
gmap.addOverlay(marker);
//End of add marker code
The editable areas in red are where you can change the coordinates of the marker and the description in the pop-up box.
I will also plot the locations of a couple aftershocks that occurred over a week after the initial event. I got these locations from the USGS Earthquake Search page at http://neic.usgs.gov/neis/epic/epic_circ.html
var point = new GLatLng(18.38,-72.85);
var marker = createMarker(point,'Magnitude 4.9 Aftershock Jan 21, 2010.');
gmap.addOverlay(marker);
var point = new GLatLng(18.48,-72.45);
var marker = createMarker(point,'Magnitude 4.4 Aftershock Jan 22, 2010.');
gmap.addOverlay(marker);
You can continue adding as many points as you like in this fashion.
You will probably want to change your markers so they don’t all look the same. In my case, I would like to distinguish the earthquake from the aftershocks. You can use any of the icons shown on this page, instead of the default marker. To do that, add this code block right before the //Adds a marker with window information block:
//javascript that sets up variables that enable the map to draw a custom icon.
var baseIcon = new GIcon();
baseIcon.iconSize=new GSize(32,32);
baseIcon.shadowSize=new GSize(56,32);
baseIcon.iconAnchor=new GPoint(16,32);
baseIcon.infoWindowAnchor=new GPoint(16,0);
var custom_icon = new GIcon(baseIcon, "http://maps.google.com/mapfiles/kml/pal3/icon28.png", null, "http://maps.google.com/mapfiles/kml/pal3/icon28s.png");
//end of custom icon code
Change the red pal group number and icon number to match the icon you want to use. Then, add the blue text to the end of these lines:
//Adds a marker with window information
function createMarker(point,html,icon) {
var marker = new GMarker(point,icon);
GEvent.addListener(marker, "click", function() {
marker.openInfoWindowHtml(html);
});
return marker;
}
var point = new GLatLng(18.44,-72.54);
var marker = createMarker(point,'Magnitude 7.0 Earthquake Jan 12, 2010.', custom_icon);
gmap.addOverlay(marker);
Note: if you wanted to use two types of icons, you could define (for example) a custom_icon2 above and assign it below, in the same manner.
It is simple to add KML files, which were developed for use with Google Earth, to these maps. Although shapefile is still the most common format, KML is becoming more and more available. My favorite new data search tool, the Geocommons Finder! offers its entire catalog in KML format (along with SHP and CSV).
To add a KML layer that shows the locations of hospitals in Haiti, you’d insert the following code right before the //Close bracket for the function initialize:
//Adds a kml overlay
var geoXml = new GGeoXml("http://finder.geocommons.com/overlays/20906.kml");
gmap.addOverlay(geoXml);
//End of kml overlay code
You can add as many of these as you like, and change the URL to point to whatever file you’d like. Note: If you, like me, have most of your data in other formats, you can use Havard’s Export to KML: Version 2.4 ArcGIS extension (see list on right side of page) before adding it to a mashup.
Here’s my Haiti map with all the changes I described. View the source to see them in context.
Tags: Google Maps API, javascript, kml, mashup, symbol
Displaying in 3D with ArcScene
Problem: You have some elevation or multipatch data that you would like to visualize in three dimensions.
For several of my entries I have been using layers from the robust catalog over at DC GIS. They offer pretty much everything needed to make a map of the District. In addition, some of the data layers contain elevation information, such as the 1 meter topo lines and the mulipatch buildings.
Note: Multipatch is a unique geometry type that can contain 3D models with vertical and overhanging faces.
These layers lend themselves to being rendered in 3D, and ArcScene has been formulated for that purpose.
ArcScene will show up in your Start Menu along with ArcMap and ArcCatalog. There is also a button that launches it from the 3D Analyst toolbar.
You can add data to ArcScene using the Add Data button just like you would to ArcMap, but it will look very different. Any of the height information embedded in the layers will automatically be applied to form a 3D world.
Here’s how the multipatch buildings, topo lines, road areas, lot boundaries, tree points and swimming pools from DC GIS look when I first add them.
It’s pretty neat, but needs some work. Some of layers, like the lots and roads, appear to be floating beneath the ground. This is because those layers don’t have height information, so ArcScene doesn’t know where to put them. In order to proceed, we need to make a surface from those topo lines. To do that, go to the 3D Analyst drop down menu, and select Create/Modify TIN –> Create TIN from Features. This will create a Triangulated Irregular Network to act as an opaque ground surface that the other layers can be draped over.
You just need to tell it which feature layer to use to create the TIN, which attribute has the height information, and which triangulation method to use. Soft Line is best for topos.
Now I have a solid ground surface with realistic shadows to enhance the 3D effect.
In order to raise up the other layers so they sit on top of the ground, right click on them to open their Properties. Then go to the Base Heights tab and tell ArcScene to obtain their heights from the surface you just created.
I did this for all my layers and also changed the symbols to some more pleasing colors. Now it’s looking like a real neighborhood!
Those multipatch buildings are by far the coolest part. They’re likely pretty rare in the typical GIS catalog, however. If you’re like most people and just have normal (boring) building footprint polygons, at this point your scene would look thusly:
You can add more realism by “extruding” the buildings up from the ground by a uniform height. Go back to the Layer Properties and click on the Extrusion tab.
Play around with different values until it looks right. I found that 8 was about the height of most of the multipatches. It just looks like they’ve all got flat roofs now.
If I wanted to, I could extrude the roads and swimming pools just a bit too. I tried it, and decided they looked better flat. I also changed the tree symbol to Big Leaf Maple (available through More Symbols –> 3D Trees), checked off the Edge Types symbol on the TIN, and turned on the lot boundaries as hollow polys. Just some ideas to show you different things you can do.
Whatever you do, you are going to want to try a fly-through. You won’t be able to resist. I sure couldn’t.
Press the button that looks like a little bird, then click in the scene where you want the bird to go, and use the mouse to soar through the streets at eye level. If you are giving a presentation, this is a guaranteed crowd pleaser. Here’s some “photos” I took while flying through my scene.
Tags: 3D, ArcScene, multipatch, TIN, topography
Viewing Flood Zones in ArcGIS Explorer, Part 2
This is a continuation of my last post about different ways to access flood zone data for the non-ArcGIS Desktop user.
Method 4: DFIRM Shapefiles
Digital Flood Rate Insurance Maps are available to download from FEMA for $10. They’ve offered a few free samples and Fairfax City happens to be one of them.
The data comes in several formats including shapefile. ArcGIS Explorer can read shapefiles. However, it will not let you add them to your map unless they have a defined projection.
The shapefiles in the Fairfax City DFIRM that I downloaded didn’t have their projections defined. I would assume this is the case with all of them. Luckily, they tell you the projection in the metadata. And luckily, projections can be defined with a file you can create using any text editor.
To find the projection, open the _metadata file in the Document folder. If you scroll down about 2/3 of the way you’ll find the Spatial_Reference_Information section. The most important parts are the Grid_Coordinate_System_Name, UTM_Zone_Number, and Horizontal_Datum_Name. The Fairfax City DFIRM is in Universal Transverse Mercator (UTM) Zone 18, NAD 1983 datum.
I used that information to have ArcGIS desktop create a projection definition file in the format used by all ESRI GIS software (including ArcGIS Explorer). It looks like this:
PROJCS["NAD_1983_UTM_Zone_18N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-75.0],PARAMETER["Scale_Factor",0.9996],
PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
All you need to do is copy and paste that text into a text editor, remove any spaces, then save it as a .prj file. The name before the file extension should match the shapefile you are trying to use. The main DFIRM shapefile is S_Fld_Haz_Ar.shp, the flood hazard zone areas. So the projection definition file should be called S_Fld_Haz_Ar.prj
If you don’t want to copy and paste you can download it and put it the same folder with the shapefile.
FEMA uses UTM for all of its DFIRMs, but they do not always use the same datum. If you download a different one from Fairfax City, you will need to check the metadata for the UTM Zone Number and whether the datum is NAD 1983 or NAD 1927. If it’s NAD 1983, you can use the same text from above as a template to create your .prj file. Just change the two red areas to match what the metadata says:
PROJCS["NAD_1983_UTM_Zone_18N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-75.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
If it’s NAD 1927, use this template and change the red areas
PROJCS["NAD_1927_UTM_Zone_17N",GEOGCS["GCS_North_American_1927",DATUM["D_North_American_1927",
SPHEROID["Clarke_1866",6378206.4,294.9786982]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-81.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
(Remove any line breaks that I’ve entered for readability).
Now, Go to the Add Content button and select Shapefiles… Then browse to the ArcShapes folder and add S_Fld_Haz_Ar.shp. The shapefile starts out looking like this…
…which isn’t very helpful, but just wait. If you right click on the layer in the Contents window, you can change the symbol to something with edges. Now you will be able to see the flood zone borders.
And, if you right click on the layer again and this time bring up the Properties window, you will be able to select certain attributes to show as Popup Content.
(click on the image to see full size)
When you open this dialog box, a list of all the available attributes will come up. You will be able to select which ones you want to appear in a little pop-up window whenever you click on a feature. I picked all of them. Then, in the bottom half of the box you can select one attribute that will appear whenever you hover over a feature with your mouse. I picked FLD_ZONE because this is the most important piece of information. Now, if I type the address of City Hall into the Find box and press enter, I have everything I wanted at the beginning.
There’s a “you are here” symbol, and if I mouse-over I see it’s located in Zone X (not Flood Zone). I can mouse over other areas to see where the nearest 0.2 pct annual chance flood hazard zone is. And if I click in the zone, I can get any more information that exists about it.
Knowing how to use shapefiles in ArcGIS Explorer opens up a world of information. You can watch a free basic overview of the software at http://blogs.esri.com/Support/blogs/esritrainingmatters/archive/2009/12/03/explore-arcgis-explorer-in-a-free-training-seminar.aspx
Tags: ArcGIS Explorer, attribute table, coordinate system, DFIRM, flood zone, shapefile
Viewing flood zones with ArcGIS Explorer, Part 1
Problem: You need to look at some GIS data without purchasing a GIS software package.
This post is inspired by a reader question. My friend Jeremy asks,
“Is there any easy way to get at GIS data of water tables, flood zones, and stuff like that? Things that people who buy houses really should look into but seldom do.”
I cut my teeth making flood zone maps so I am quite familiar with where this data lives. I revisited the FEMA Map Service Center to give him some specific advice about where to look. My plan was to determine the easiest/fastest path to the information. To my dismay, the website has not changed much since I changed jobs. Meaning, to put it bluntly: there is no easy way.
I assume most visitors to the website would want to be able to type in an address and have that address come up as a dot on a map with the flood zone layer below it. If the dot falls within a flood zone, they’d be able to tell in one glance. Well, you can get pretty close, but you can’t exactly do that. So, instead of talking about the easiest way to get flood zone information, I’ll talk about four different ways.
I attended college at George Mason University so I decided to use the city of Fairfax as my example location. Pretend you are moving there. Because I don’t want to map a private residence, pretend you are moving there because you just got elected mayor. You want to check if your new digs at City Hall are in danger of flooding.
Method 1: FEMA’s Map Viewer
https://hazards.fema.gov/wps/portal/mapviewer
Pros
- This viewer allows you to type in an address and zoom to it.
- The map that comes up shows the flood zones in that area along with a lot of other useful information.
- There’s an identify tool that allows you access even more information about the data layers.
Here’s what I got for City Hall:
Cons
- There’s no “you are here” symbol on the map. If I’m not familiar enough with the area to already know where City Hall is, I still don’t know whether it’s in a flood zone.
- There’s too many layers turned on by default, which clutters the map, making it confusing and hard to interpret at first glance.
- The legend isn’t very helpful. It doesn’t explain what the blue-hatched Zone A is, or what the purple text LOMRs are. I only know that they are Letters of Map Revision because I used to work there.
Method 2: Product Search by Address…
Box on top-left corner of main Map Service Center screen
You can enter an address and select either a Flood Map (visual image) or DFIRM (digital database) product. The Fairfax City data is at the moment only available as a DFIRM, which I will discuss later in method 4. For now, I’ll enter the address of nearby GMU campus to bring up the Flood Map of surrounding Fairfax County.
Pros
- A beautiful looking map (If I don’t say so myself) which is simple and easy to interpret, and also happens to be the authoritative flood zone representation product.
- A good legend which explains in detail what the flood zone designations mean.
Cons
- Obviously no “you are here” symbol on this static map.
- And, it’s even harder to pinpoint an exact location than in the Map Viewer, because these maps only show major roads, and most of the time use route numbers instead of road names.
It isn’t hard to see why people may feel the need to look elsewhere to determine if their prospective property is really in a flood zone. Maybe that’s why a google search for “flood zone data” brings up companies that will research this for you if you pay them. There are, however, other ways, which are free or almost free.
Method 3: FEMA’s Web Map Service (WMS) & ArcGIS Explorer
https://hazards.fema.gov/femaportal/wps/portal/NFHLWMSe
A web map service is a computer program that will query a database and deliver the results back in the form of a dynamically generated map. FEMA makes the National Flood Hazard Layer available for free through a WMS. You need to have GIS software in order to read WMS data. However, if you don’t have access to the ArcGIS Desktop package that I regularly blog about, you aren’t locked out from playing. You just need to grab a copy of ArcGIS Explorer: “GIS for Everyone” at http://www.esri.com/software/arcgis/explorer/index.html
Once you’ve installed and opened the program, press the Add Content icon on the main toolbar. Select GIS Services… and then press New Server Connection. For the server type, chose WMS, and then enter the Server address that FEMA gives on their tool description page:
http://hazards.fema.gov/wmsconnector/wmsconnector/Servlet/NFHL?
You’ll then be given a list of services to choose from. Flood Hazard Zones (General) is the main one. If you pick that, then enter your address in the Find box on the left side of the screen, the globe will whirl around until you see this.
Eureka! Now I know City Hall isn’t in a flood zone!
(I changed the Basemap to Streets)
Stay tuned, in two weeks I will post about Method 4: Using DFIRM Shapefiles. The best is yet to come.
Tags: ArcGIS Explorer, flood zone, Web Map Service
Multi-line text in legends
Problem: You need to create a legend with long descriptions that wrap to more than one line
Last week I read the Strange Maps slideshow essay on Slate.com, and fell in love with the last map: Soil Map of the Central Far Side of the Moon. The interesting colors make it look like an abstract painting. I stared at it appreciatively for only a little while before I began to wonder what each shade represented.
This map is available from the USGS at http://astrogeology.usgs.gov/Projects/PlanetaryMapping/DIGGEOL/moon/1047/lfar.htm. Unfortunately, the .pdf file on that web page does not have accompanying explanatory text, either. I had to dig futher down to find it, spread between two separate pages: a chart that links color swatches to soil type codes, and a list that links soil type codes to full descriptions. If all of this information was displayed together on one page as a map and legend, it would be an even more captivating piece.
Producing such a legend for an ArcMap layout is kind of tricky. ArcMap does not, by default, handle color swatches along with multiple lines of text very well. What you would envision looking something like this…
…turns out looking something like this…
….with no line breaks, and no good way to create them. I arrived at the bad-looking purple legend item in the typical way you would think to make a legend. That is, I created an attribute table field for Soil Descriptions. I called that field “Desc_” and I copied the long text descriptions into it. Then, I symbolized by category on the Desc_ field. My symbolgy definition box looked like this:
You can already start to see the problem. The Label area is truncating, and that’s where ArcMap looks to get the legend text. Yes, you can double click in this area to edit the text, but you will not be able to enter line breaks.
One solution is to convert your entire legend to graphics, ungroup it, and start editing the text boxes manually. That will work, but then you will have a static legend that doesn’t update with changes to your map. Do it a couple times, and then have to remake your entire legend when you’re asked to add new items or use a different color scheme, and you won’t want to do it again.
The solution I will describe takes more time to set up initially, but will make your life easier later on. The answer is to make ArcMap take the legend text from a special Description area instead. The Description box will let you use line breaks! But, it is not available if you symbolize by category. It is only available if you use the “draw all features using the same symbol” option. Luckily, you can mimic the effect of symbolizing by category by using multiple versions of the same layer with definition queries applied.
So, to create a legend item for the lavender “It” soil that looks like the legend item for the brown “NpNt” soil, you would follow the steps below.
1. Add the soil layer to ArcMap a second time.
2. Create a definition query on the new layer so it only displays soil type “It”.
3. Symbolize all features with the lavender color.
4. Copy the soil description text into Notepad or any text editor, and enter line breaks where you want them. Ideally you’d do this for all your soil descriptions at once so you know they’re all the same width.
5. Bring up the Layer Properties, Symbology tab. Press the Description… button and copy the text from Notepad into the Description for Legend box.
Note that you cannot create line breaks in this box by pressing the enter key! But, any line breaks you have created in Notepad will be maintained by copying and pasting.
6. Go to Layout view and insert a new legend (or right click on the one that’s there) to bring up the Legend Properties. Go to the Items tab, select your newest entry, and press the Style… button.
7. Pick one of the styles that uses “Description” instead of the default “Label” for the swatch text.
Finally, it’s looking pretty good!
You can repeat the same process for as many legend items as you need. In the end, the legend will appear and behave just as if you had symbolized by category.
If you want to take a little more time, you can make it look even better. Most people envision these sorts of legends with the swatch justified with the top of the text. You can do that if you drill down a bit within the settings. Return to step #6 where you selected the legend item and pressed the Style… button. This time, press Properties… in the Legend Item Selector box that comes up. If you go the General tab you’ll see this:
Show Descriptions should be checked because of the changes you made earlier. Press the Description Symbol… button below it. Then, press Properties… again in the Symbol Selector window that comes up. Change the Vertical Alignment to Top.
That’s shaping up into one respectable legend. 2 soils down, about 25 to go. I’m half a mind to complete it just so I can stare at that map a little longer.
Tags: dynamic label, legend, line break
Tied Together (with GIS)
I took last week off from composing a new entry in honor of Thanksgiving. Instead, please enjoy this music video some Austin, Texas GIS users created on GIS Day. Consider it a fun multimedia answer to the “What is GIS?” question.
Saving symbology to use again
Problem: You have encountered or created some really nice symbols and want to stash them away for future use.
Sometimes it takes forever to get a symbol just right. And sometimes you’re looking at a map or layer file made by someone else and just love what they’ve done. In either case, it is not necessary to repeat the work, because you can save symbology definitions in a couple different ways.
The ESRI Streetmap package comes with a layer file that can be used to produce pretty maps from the data. In particular, I like the orange star symbol they use for state capitals.
In order to save it for re-use, I would follow the steps below. These same steps also can be used to save line symbols, fill symbols (for polygons), and font definitions (for text labels).
Bring up the Layer Properties, Symbology tab, and double click on the symbol to open the Symbol Selector. (For text labels you get to the Symbol Selector by going the Labels tab and pressing the Symbol… button). On the symbol selector screen there is a Save… button. If you press it, it will ask you to give the symbol a name and a category.
That’s really all there is to it! Your saved symbol will now appear at the top of the list of symbol choices. It will be there whenever you open ArcMap from now on, available for use like any other symbol. In addition, if you go to the Category drop down you will be able to see only the symbols in the category you just created. So, for instance, I could save all the ESRI Streetmap symbols I like, and give them that same category name to make them easy to find later. (Note: The new category may not show up right away, but if you open and close ArcMap it will be there next time).
So, how does ArcMap save these symbols and where does it look for them next time? It saves them into something called a .style file. Every custom symbol defintion that you save is stored in your personal style file, located in \ApplicationData\ESRI\ArcMap\username.style. The ESRI styles are stored in \Program Files\ArcGIS\Styles. Both your personal style and the default ESRI style load every time ArcMap does. You can access the other ESRI styles by clicking on the More Symbols button in the symbol selector.
Just like layer files, you can manipulate these style files and share them with others. I found a cool one that Brady Davis shared back in 2005 to use as an example. He created a style based on Google Maps. You can download it at: http://conversationswithmyself.com/181/ Once you unpack it, you can add it to ArcMap by pressing the More Symbols button and selecting Add…
You can get to all the new Google symbols by navigating through the symbol selector for different symbol types. But if you’d like to take a quick peek at everything all at once, that’s what the Style Manager is for.
You can open the Style Manager by going to Tools –> Styles –> Style Manager in the main ArcMap interface. You will see your personal style there, the ESRI default style, and any other styles that you may have added or turned on. Here’s how the Google style looks:
You can see from the folder names that you can control much more than just point, line, fill and text symbols with the style manager. Folders are colored yellow if they have something in them.
Also, with the style manager, you can create new styles, and copy and paste elements from one style into another. So, you could copy all your favorite symbols into you personal style. Or, you could create and populate a style specifically for sharing with others. You can also delete elements you don’t like to reduce clutter.
In addition, Tools –> Style –> Style References lets you pick which styles will show up in the symbol selector window. So, you could create an mxd that only references a custom style you have created, thereby limiting the user’s options to a list of standard symbols.
So, with just a little tweaking I used the Google style to googleize a section of downtown Richmond.
Not too shabby! That’s the power of styles.
Tags: style manager, symbol
Enabling and editing z-values
Problem: You need to load a shapefile that does not have z-values into a feature class that does
Some spatial data will store height information in a normal attribute table field, but others will store it in the feature geometry as a z-value. Using z-values increases the feature geometry from 2D (x and y) to 3D (x,y and z). ArcScene will automatically display 3D data using the z-values, and it will do so more rapidly than draping the data over a surface.
Because z-values are optional, you may run into a situation where you need to merge a layer that does not have them into a layer that does. For example, say you have a geodatabase holding tree locations along neighborhood streets. The height of the trees is stored in z. Your surveyors have gone out and collected the locations of some additional trees in people’s yards, but they didn’t use 3D geometry. You need to load the new yard trees into the street tree feature class.
Right clicking and selecting properties in ArcCatalog confirms that the StreetTreePntZ feature class has z-values but the YardTree shapefile does not.
Trying to load the shapefile as-is will result in the error message.
One solution: you could run ET GeoWizards Point to PointZ conversion on the shapefile.
Another solution: Environment Settings! Import the yard tree shapefile into your geodatabase as a separate feature class. But before you press the OK button, press Environments… right next to it.
Expand General Settings and under Output has Z-Values, change Same as Input (the default) to Enabled.
Now you will be able to load the z-enabled yard tree feature class into the StreetTreePntZ feature class.
The z-values for your newly loaded features will be 0. You can edit z values in ArcMap by starting an edit session, double clicking on each feature (to turn it into a sketch) and then opening the Sketch Properties dialog from the Editor toolbar.
In the future I will write an entry about visualizing these cool 3D features in ArcScene.
Tags: 3D, data loader, settings, z value
Georeferencing for Vectors: Overview of Spatial Adjustment
Problem: Vector datasets are not lining up correctly in ArcMap, and you don’t know their projection
At the end of my post about defining the projection of AutoCAD files, I promised to write about Spatial Adjustment. Spatial adjustment is much like georeferencing an image, wherein you place control points at known common locations between the target layer and a reference layer, then run a process to align the two. ArcMap’s spatial adjustment toolbar allows you to do the same thing to a vector dataset using a slightly different, but familiar workflow. Please note that spatial adjustment should only be used when it is impossible to determine the projection of your vector data. Defining the correct projection will result in a much more accurate result.
I am returning to the city of Bloomington, Indiana and their AutoCAD files for this example. I’ve downloaded the landuse zoning dxf and the municipal boundary shp. I then projected the shapefile to a different coordinate system and cleared the projection definition so ArcMap is putting it in the wrong place. We’ll pretend you’ve received this shapefile and nobody has told you its projection. You need it to line up with the zoning AutoCAD layer.
It is more likely you would be doing things the other way around, that is, that you would have received AutoCAD data with an unknown projection. However, spatial adjustment only works while inside an edit session. You cannot “start editing” AutoCAD data. You would need to convert it to a shapefile or feature class before you could spatially adjust it.
That brings us to step one. Start an edit session. Then, go to View –> Toolbars and check on the Spatial Adjustment toolbar if it is not visible already. Use the main dropdown menu to Set Adjust Data and chose your target layer (in this case the municipal boundary shape). You will have the option to adjust all the features or only the selected features. We will do all of them.
Then, press the button that looks just like Georeferencing’s “Add Control Points” to add your first Displacement Link. The lingo is different; the concept is the same.
Click on a distinguishing feature on the municipal boundary and then on that same feature in the zoning layer. Remember, you always start with the layer that you want to move and end by telling it where to go.
One difference that takes some getting used to is that there is no Shift or Fit to Display ability like there is in georeferencing. So, you can’t get the target layer and the reference layer into the same window before you start adding displacement links. You will have to keep zooming back and forth between them. If you can’t place your links as precisely as you’d like due to this, don’t worry, you can get them in the general vicinity and then press Modify Link to edit them.
Another difference between this and georeferencing is that placing links doesn’t automatically cause the layer to move. That only happens when you select Adjust from the dropdown menu. You will not be able to select Adjust until you have placed three links. I recommend doing so as soon as you can just to get things into the same universe. (There’s no limit to how many times you can re-adjust later.) Just like with georeferencing, place your three links as far apart from each other as possible.
Once you have adjusted, your links will disappear, and you will be able to refine your results by adding more links.
(After first adjustment)
At this point you may also want to experiment with different adjustment methods. The first three in the list are coordinate system transformations. You should stick with these when the only reason your data isn’t lining up is that you don’t know the projection.
- Affine (the default) will scale, skew, rotate, and translate the data.
- Similarity will scale, rotate, and translate but it will not skew. It will not change the aspect ratio or the shape of features– only their size, position and orientation.
- Project uses a more complex formula that is tailored to features that have been captured from aerial photography.
The last two should only be used if you know there are errors within your target layer that make it less accurate than your reference layer.
- Rubbersheeting behaves like it sounds– pulling and stretching your layer in a piecewise fashion. The areas that are closer to displacement links will move more dramatically. Identity links (the black and red square crosshair button) can be used to nail down features that should not move. If rubbersheeting is used it should be as a second step after transformation.
- Edge Matching is used to merge adjacent layers that have been split up. It makes sure features line up along one edge only.
You can preview the results of different adjustment methods in the preview window. You can also use the preview window to see the effects different displacement links will have.
Another way to check your work is by looking at the link table. If you are using a transformation adjustment method, get an idea of your RMS error. If you place a link that increases the RMS and Residual errors substantially, delete the link and try something else. I recommend keeping both the Preview Window and the Link Table open while you are working.
When you are happy with what you see, make a final adjustment and then save your edits.
Tags: coordinate system, spatial adjustment
Using ET GeoWizards to enhance shapefile management
This week I am writing about yet another plugin that has proven quite useful to me. I have only scratched the surface of its capabilities, but the few features I did try out, I found reasons to use over and over. The plugin is ET GeoWizards, developed by Ianko Tchoukanski, and available at http://www.ian-ko.com/
This tool duplicates some of the capabilities already present in ArcGIS, however it makes them all available at the ArcView license level. This can be a real boon to people who don’t have ArcEditor or ArcInfo licenses. In addition, you might just prefer the way it handles things better.
For example, my favorite function is the most basic: “Create New Shapefile.” I love it because it lets me create new shapefiles right in ArcMap! No need to disrupt my workflow to start up ArcCatalog and then drag the file into ArcMap so I can start editing it. When you initiate shapefile creation, it lets you chose a spatial reference based on your current map or other layers.
Then, it lets you add attribute fields! This too would be a separate step doing it the ArcCatalog way.
Then, it dumps your new shapefile right into the Table of Contents, ready for you to start using it. This is so much more convenient that you will never want to go back to the old way. It alone is reason to get the plugin, but there is more.
My second favorite function is “Redefine Fields,” which lets you change the length of string fields, or the precision of number fields, in your attribute table. I have run into many cases where I needed my text fields to be longer, and this is the easy answer. The only way to increase field length otherwise is to delete the field and re-add it with different definitions. This is problematic if the field is already populated with data. You end up having to create “holder cells” and migrate the data back in.
This tool removes the need for all those intermediate steps. However, it does save the results into a new shapefile, instead of updating your existing shapefile. That isn’t ideal, but I think there’s no way around it. It is still a much simpler solution overall.
My third favorite function is “Order Fields” which changes the order of the fields in your attribute table. There is no other way to do this. Sometimes, you want the most important information to be in front, especially if you have lots of fields to sort through. With ArcMap, you can drag fields to reorder them, but they snap back to their original position once you close the attribute table. This tool changes their order for good. Also, if you want to remove any fields while you’re at it, you can do so by leaving them over on the left side. Like “Redefine Fields”, it saves the results into a new shapefile.
A couple other functions that I haven’t tried, but which look really good:
- “Generalize,” which reduces the number of vertices used to represent a polyline or polygon. Sometimes you will end up with a feature you need to change the shape of that has vertices packed so tightly you are going to be there all day dragging things. Problem solved!
- “Shape to ShapeZ” conversion, which will add the Z dimension to a shapefile. Z values allow for the storage of elevation data. You can’t load Z-enabled data into non Z-enabled feature classes. You have to drill down into the Environment Settings in order to enable Z values. It’s tricky enough that I will probably blog about it at some point. This looks to be an easier way.
As I said above, I have only begun to explore the free ET GeoWizards functions. The registered version has even more capabilities. And, there are ET GeoTools for inline editing and ET Surface tools for working with raster elevation data. Plenty of reasons to see what’s at http://www.ian-ko.com
Tags: attribute table, shapefile