Archive for the ‘Tools’ Category
Viewing Flood Zones in ArcGIS Explorer, Part 2
This is a continuation of my last post about different ways to access flood zone data for the non-ArcGIS Desktop user.
Method 4: DFIRM Shapefiles
Digital Flood Rate Insurance Maps are available to download from FEMA for $10. They’ve offered a few free samples and Fairfax City happens to be one of them.
The data comes in several formats including shapefile. ArcGIS Explorer can read shapefiles. However, it will not let you add them to your map unless they have a defined projection.
The shapefiles in the Fairfax City DFIRM that I downloaded didn’t have their projections defined. I would assume this is the case with all of them. Luckily, they tell you the projection in the metadata. And luckily, projections can be defined with a file you can create using any text editor.
To find the projection, open the _metadata file in the Document folder. If you scroll down about 2/3 of the way you’ll find the Spatial_Reference_Information section. The most important parts are the Grid_Coordinate_System_Name, UTM_Zone_Number, and Horizontal_Datum_Name. The Fairfax City DFIRM is in Universal Transverse Mercator (UTM) Zone 18, NAD 1983 datum.
I used that information to have ArcGIS desktop create a projection definition file in the format used by all ESRI GIS software (including ArcGIS Explorer). It looks like this:
PROJCS["NAD_1983_UTM_Zone_18N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-75.0],PARAMETER["Scale_Factor",0.9996],
PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
All you need to do is copy and paste that text into a text editor, remove any spaces, then save it as a .prj file. The name before the file extension should match the shapefile you are trying to use. The main DFIRM shapefile is S_Fld_Haz_Ar.shp, the flood hazard zone areas. So the projection definition file should be called S_Fld_Haz_Ar.prj
If you don’t want to copy and paste you can download it and put it the same folder with the shapefile.
FEMA uses UTM for all of its DFIRMs, but they do not always use the same datum. If you download a different one from Fairfax City, you will need to check the metadata for the UTM Zone Number and whether the datum is NAD 1983 or NAD 1927. If it’s NAD 1983, you can use the same text from above as a template to create your .prj file. Just change the two red areas to match what the metadata says:
PROJCS["NAD_1983_UTM_Zone_18N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-75.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
If it’s NAD 1927, use this template and change the red areas
PROJCS["NAD_1927_UTM_Zone_17N",GEOGCS["GCS_North_American_1927",DATUM["D_North_American_1927",
SPHEROID["Clarke_1866",6378206.4,294.9786982]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-81.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
(Remove any line breaks that I’ve entered for readability).
Now, Go to the Add Content button and select Shapefiles… Then browse to the ArcShapes folder and add S_Fld_Haz_Ar.shp. The shapefile starts out looking like this…
…which isn’t very helpful, but just wait. If you right click on the layer in the Contents window, you can change the symbol to something with edges. Now you will be able to see the flood zone borders.
And, if you right click on the layer again and this time bring up the Properties window, you will be able to select certain attributes to show as Popup Content.
(click on the image to see full size)
When you open this dialog box, a list of all the available attributes will come up. You will be able to select which ones you want to appear in a little pop-up window whenever you click on a feature. I picked all of them. Then, in the bottom half of the box you can select one attribute that will appear whenever you hover over a feature with your mouse. I picked FLD_ZONE because this is the most important piece of information. Now, if I type the address of City Hall into the Find box and press enter, I have everything I wanted at the beginning.
There’s a “you are here” symbol, and if I mouse-over I see it’s located in Zone X (not Flood Zone). I can mouse over other areas to see where the nearest 0.2 pct annual chance flood hazard zone is. And if I click in the zone, I can get any more information that exists about it.
Knowing how to use shapefiles in ArcGIS Explorer opens up a world of information. You can watch a free basic overview of the software at http://blogs.esri.com/Support/blogs/esritrainingmatters/archive/2009/12/03/explore-arcgis-explorer-in-a-free-training-seminar.aspx
Tags: ArcGIS Explorer, attribute table, coordinate system, DFIRM, flood zone, shapefile
Viewing flood zones with ArcGIS Explorer, Part 1
Problem: You need to look at some GIS data without purchasing a GIS software package.
This post is inspired by a reader question. My friend Jeremy asks,
“Is there any easy way to get at GIS data of water tables, flood zones, and stuff like that? Things that people who buy houses really should look into but seldom do.”
I cut my teeth making flood zone maps so I am quite familiar with where this data lives. I revisited the FEMA Map Service Center to give him some specific advice about where to look. My plan was to determine the easiest/fastest path to the information. To my dismay, the website has not changed much since I changed jobs. Meaning, to put it bluntly: there is no easy way.
I assume most visitors to the website would want to be able to type in an address and have that address come up as a dot on a map with the flood zone layer below it. If the dot falls within a flood zone, they’d be able to tell in one glance. Well, you can get pretty close, but you can’t exactly do that. So, instead of talking about the easiest way to get flood zone information, I’ll talk about four different ways.
I attended college at George Mason University so I decided to use the city of Fairfax as my example location. Pretend you are moving there. Because I don’t want to map a private residence, pretend you are moving there because you just got elected mayor. You want to check if your new digs at City Hall are in danger of flooding.
Method 1: FEMA’s Map Viewer
https://hazards.fema.gov/wps/portal/mapviewer
Pros
- This viewer allows you to type in an address and zoom to it.
- The map that comes up shows the flood zones in that area along with a lot of other useful information.
- There’s an identify tool that allows you access even more information about the data layers.
Here’s what I got for City Hall:
Cons
- There’s no “you are here” symbol on the map. If I’m not familiar enough with the area to already know where City Hall is, I still don’t know whether it’s in a flood zone.
- There’s too many layers turned on by default, which clutters the map, making it confusing and hard to interpret at first glance.
- The legend isn’t very helpful. It doesn’t explain what the blue-hatched Zone A is, or what the purple text LOMRs are. I only know that they are Letters of Map Revision because I used to work there.
Method 2: Product Search by Address…
Box on top-left corner of main Map Service Center screen
You can enter an address and select either a Flood Map (visual image) or DFIRM (digital database) product. The Fairfax City data is at the moment only available as a DFIRM, which I will discuss later in method 4. For now, I’ll enter the address of nearby GMU campus to bring up the Flood Map of surrounding Fairfax County.
Pros
- A beautiful looking map (If I don’t say so myself) which is simple and easy to interpret, and also happens to be the authoritative flood zone representation product.
- A good legend which explains in detail what the flood zone designations mean.
Cons
- Obviously no “you are here” symbol on this static map.
- And, it’s even harder to pinpoint an exact location than in the Map Viewer, because these maps only show major roads, and most of the time use route numbers instead of road names.
It isn’t hard to see why people may feel the need to look elsewhere to determine if their prospective property is really in a flood zone. Maybe that’s why a google search for “flood zone data” brings up companies that will research this for you if you pay them. There are, however, other ways, which are free or almost free.
Method 3: FEMA’s Web Map Service (WMS) & ArcGIS Explorer
https://hazards.fema.gov/femaportal/wps/portal/NFHLWMSe
A web map service is a computer program that will query a database and deliver the results back in the form of a dynamically generated map. FEMA makes the National Flood Hazard Layer available for free through a WMS. You need to have GIS software in order to read WMS data. However, if you don’t have access to the ArcGIS Desktop package that I regularly blog about, you aren’t locked out from playing. You just need to grab a copy of ArcGIS Explorer: “GIS for Everyone” at http://www.esri.com/software/arcgis/explorer/index.html
Once you’ve installed and opened the program, press the Add Content icon on the main toolbar. Select GIS Services… and then press New Server Connection. For the server type, chose WMS, and then enter the Server address that FEMA gives on their tool description page:
http://hazards.fema.gov/wmsconnector/wmsconnector/Servlet/NFHL?
You’ll then be given a list of services to choose from. Flood Hazard Zones (General) is the main one. If you pick that, then enter your address in the Find box on the left side of the screen, the globe will whirl around until you see this.
Eureka! Now I know City Hall isn’t in a flood zone!
(I changed the Basemap to Streets)
Stay tuned, in two weeks I will post about Method 4: Using DFIRM Shapefiles. The best is yet to come.
Tags: ArcGIS Explorer, flood zone, Web Map Service
Using ET GeoWizards to enhance shapefile management
This week I am writing about yet another plugin that has proven quite useful to me. I have only scratched the surface of its capabilities, but the few features I did try out, I found reasons to use over and over. The plugin is ET GeoWizards, developed by Ianko Tchoukanski, and available at http://www.ian-ko.com/
This tool duplicates some of the capabilities already present in ArcGIS, however it makes them all available at the ArcView license level. This can be a real boon to people who don’t have ArcEditor or ArcInfo licenses. In addition, you might just prefer the way it handles things better.
For example, my favorite function is the most basic: “Create New Shapefile.” I love it because it lets me create new shapefiles right in ArcMap! No need to disrupt my workflow to start up ArcCatalog and then drag the file into ArcMap so I can start editing it. When you initiate shapefile creation, it lets you chose a spatial reference based on your current map or other layers.
Then, it lets you add attribute fields! This too would be a separate step doing it the ArcCatalog way.
Then, it dumps your new shapefile right into the Table of Contents, ready for you to start using it. This is so much more convenient that you will never want to go back to the old way. It alone is reason to get the plugin, but there is more.
My second favorite function is “Redefine Fields,” which lets you change the length of string fields, or the precision of number fields, in your attribute table. I have run into many cases where I needed my text fields to be longer, and this is the easy answer. The only way to increase field length otherwise is to delete the field and re-add it with different definitions. This is problematic if the field is already populated with data. You end up having to create “holder cells” and migrate the data back in.
This tool removes the need for all those intermediate steps. However, it does save the results into a new shapefile, instead of updating your existing shapefile. That isn’t ideal, but I think there’s no way around it. It is still a much simpler solution overall.
My third favorite function is “Order Fields” which changes the order of the fields in your attribute table. There is no other way to do this. Sometimes, you want the most important information to be in front, especially if you have lots of fields to sort through. With ArcMap, you can drag fields to reorder them, but they snap back to their original position once you close the attribute table. This tool changes their order for good. Also, if you want to remove any fields while you’re at it, you can do so by leaving them over on the left side. Like “Redefine Fields”, it saves the results into a new shapefile.
A couple other functions that I haven’t tried, but which look really good:
- “Generalize,” which reduces the number of vertices used to represent a polyline or polygon. Sometimes you will end up with a feature you need to change the shape of that has vertices packed so tightly you are going to be there all day dragging things. Problem solved!
- “Shape to ShapeZ” conversion, which will add the Z dimension to a shapefile. Z values allow for the storage of elevation data. You can’t load Z-enabled data into non Z-enabled feature classes. You have to drill down into the Environment Settings in order to enable Z values. It’s tricky enough that I will probably blog about it at some point. This looks to be an easier way.
As I said above, I have only begun to explore the free ET GeoWizards functions. The registered version has even more capabilities. And, there are ET GeoTools for inline editing and ET Surface tools for working with raster elevation data. Plenty of reasons to see what’s at http://www.ian-ko.com
Tags: attribute table, shapefile
Using shpchk to fix damaged shapefiles
While I was working on this week’s blog entry, I opened an old MXD, and noticed that one of the layers in the table of contents had “gone bad.” The checkbox had grayed out and there was a red exclamation point beside it. This is a normal occurrence when a dataset gets renamed or moved to a different location, and can be repaired by updating the link. But in this case, I hadn’t been moving any files around. I attempted to “refresh” the link, by re-pointing it to the same data source, but was stopped in my tracks by this error message.
Detour! I know a way to solve this problem, so this week, instead, I will be blogging about shapechk.exe
Problem: You get a cryptic error message when trying to load a shapefile.
Shapefiles sometimes get corrupted. The errors they produce as a result used to be even less descriptive than what you see above. But as that message states, it has something to do with there being more or less entries in the attribute table (dbf) than there are in the index of spatial geometries (shp). I don’t know how it happens, but I can tell you that it does happen often enough that you will encounter it eventually.
A long time ago, Andrew Williamson wrote a utility that he doesn’t support anymore, but still offers on the web at http://www.geocities.com/SiliconValley/Haven/2295/howto_shapechk.html Note: GeoCities is closing on Oct 26th (tomorrow). After that, you can get it at ArcScripts.
It still works. It may not solve all your shapefile problems, but, up to now, it has solved all of mine. It’s a standalone executable that you simply unzip and run. Then, you press the buttons starting with Shapefile… on the left.
Browse to your damaged shapefile, then press Build Shx. Shx is the shapefile index. It will generate a new one in case your old one was the problem. Next, Check DBF.
This is the part where where it checks if the number of records in the dbf matches the number of records in the shp. If you lost your dbf entirely, it will build you a new empty one. You won’t have attributes but at least you will be able to see your shapes. In my case, the utility needed to insert 28 empty records into my dbf table to fix my shapefile.
When I opened my MXD back up, everything was working again.
Randomly placing points with Hawth’s Tools
Problem: You need to position a specific number of points in random locations within a polygon
Randomization is sometimes needed for environmental sampling in the field. In my case I needed to produce maps of random soil sample locations within property boundaries. There is no easy way within ArcGIS to accomplish such a thing. Luckily, it is one of the many capabilities included with the fantastic Hawth’s Tools plugin. This plugin is available for free at http://www.spatialecology.com/htools/
Let’s put it to work in an ordinary neighborhood such as the one pictured below.
First, download Hawth’s Tools and run the setup package. Then, add it to your ArcMap toolbar by right clicking anywhere in the empty space, selecting Customize, and checking on Hawth’s Tools in the list of available toolbars. Before you run the tool, zoom in on your property and select it (indicated by the cyan outline). Once the parcel polygon is selected, chose Sampling Tools –> Generate Random Points from the HawthsTools drop-down menu.
A dialog box will come up in which you’ll need to specify:
- The Reference layer (layer within which you want to place the points). In this example that’s OwnerPLY, my parcel boundary layer. Check “Use selected features only” to limit the output to the single property you selected previously.
- Any layers where you don’t want points to show up. In this case, it isn’t feasible to take soil samples under the house, so I’ve told the tool to prevent points from occurring in BldgPly.
- Minimum distance between points, if you want that. I didn’t specify it here.
- The number of points to generate. Five in this example. Note: stratified sampling would create your specified number of random points in every parcel polygon.
- The name & location of your output shapefile.
The tool will generate five points following all those rules. It will add an ID column to the attribute table, filled with zeros. It’s an easy matter to change those zeros to numbered point IDs. Then, if you run your new shapefile through ArcToolbox’s Add XY Coordinates, and turn labels on for the ID field, you end up with this.
Give the field workers that map and table of coordinates, and they have what they need to go out and collect those samples. By the way, Hawth’s first Sampling tool: Create Random Selection, could be used to randomly select one of these 5 random points, for double duty.