Posts Tagged ‘coordinate system’
Viewing Flood Zones in ArcGIS Explorer, Part 2
This is a continuation of my last post about different ways to access flood zone data for the non-ArcGIS Desktop user.
Method 4: DFIRM Shapefiles
Digital Flood Rate Insurance Maps are available to download from FEMA for $10. They’ve offered a few free samples and Fairfax City happens to be one of them.
The data comes in several formats including shapefile. ArcGIS Explorer can read shapefiles. However, it will not let you add them to your map unless they have a defined projection.
The shapefiles in the Fairfax City DFIRM that I downloaded didn’t have their projections defined. I would assume this is the case with all of them. Luckily, they tell you the projection in the metadata. And luckily, projections can be defined with a file you can create using any text editor.
To find the projection, open the _metadata file in the Document folder. If you scroll down about 2/3 of the way you’ll find the Spatial_Reference_Information section. The most important parts are the Grid_Coordinate_System_Name, UTM_Zone_Number, and Horizontal_Datum_Name. The Fairfax City DFIRM is in Universal Transverse Mercator (UTM) Zone 18, NAD 1983 datum.
I used that information to have ArcGIS desktop create a projection definition file in the format used by all ESRI GIS software (including ArcGIS Explorer). It looks like this:
PROJCS["NAD_1983_UTM_Zone_18N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-75.0],PARAMETER["Scale_Factor",0.9996],
PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
All you need to do is copy and paste that text into a text editor, remove any spaces, then save it as a .prj file. The name before the file extension should match the shapefile you are trying to use. The main DFIRM shapefile is S_Fld_Haz_Ar.shp, the flood hazard zone areas. So the projection definition file should be called S_Fld_Haz_Ar.prj
If you don’t want to copy and paste you can download it and put it the same folder with the shapefile.
FEMA uses UTM for all of its DFIRMs, but they do not always use the same datum. If you download a different one from Fairfax City, you will need to check the metadata for the UTM Zone Number and whether the datum is NAD 1983 or NAD 1927. If it’s NAD 1983, you can use the same text from above as a template to create your .prj file. Just change the two red areas to match what the metadata says:
PROJCS["NAD_1983_UTM_Zone_18N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-75.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
If it’s NAD 1927, use this template and change the red areas
PROJCS["NAD_1927_UTM_Zone_17N",GEOGCS["GCS_North_American_1927",DATUM["D_North_American_1927",
SPHEROID["Clarke_1866",6378206.4,294.9786982]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-81.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
(Remove any line breaks that I’ve entered for readability).
Now, Go to the Add Content button and select Shapefiles… Then browse to the ArcShapes folder and add S_Fld_Haz_Ar.shp. The shapefile starts out looking like this…
…which isn’t very helpful, but just wait. If you right click on the layer in the Contents window, you can change the symbol to something with edges. Now you will be able to see the flood zone borders.
And, if you right click on the layer again and this time bring up the Properties window, you will be able to select certain attributes to show as Popup Content.
(click on the image to see full size)
When you open this dialog box, a list of all the available attributes will come up. You will be able to select which ones you want to appear in a little pop-up window whenever you click on a feature. I picked all of them. Then, in the bottom half of the box you can select one attribute that will appear whenever you hover over a feature with your mouse. I picked FLD_ZONE because this is the most important piece of information. Now, if I type the address of City Hall into the Find box and press enter, I have everything I wanted at the beginning.
There’s a “you are here” symbol, and if I mouse-over I see it’s located in Zone X (not Flood Zone). I can mouse over other areas to see where the nearest 0.2 pct annual chance flood hazard zone is. And if I click in the zone, I can get any more information that exists about it.
Knowing how to use shapefiles in ArcGIS Explorer opens up a world of information. You can watch a free basic overview of the software at http://blogs.esri.com/Support/blogs/esritrainingmatters/archive/2009/12/03/explore-arcgis-explorer-in-a-free-training-seminar.aspx
Tags: ArcGIS Explorer, attribute table, coordinate system, DFIRM, flood zone, shapefile
Georeferencing for Vectors: Overview of Spatial Adjustment
Problem: Vector datasets are not lining up correctly in ArcMap, and you don’t know their projection
At the end of my post about defining the projection of AutoCAD files, I promised to write about Spatial Adjustment. Spatial adjustment is much like georeferencing an image, wherein you place control points at known common locations between the target layer and a reference layer, then run a process to align the two. ArcMap’s spatial adjustment toolbar allows you to do the same thing to a vector dataset using a slightly different, but familiar workflow. Please note that spatial adjustment should only be used when it is impossible to determine the projection of your vector data. Defining the correct projection will result in a much more accurate result.
I am returning to the city of Bloomington, Indiana and their AutoCAD files for this example. I’ve downloaded the landuse zoning dxf and the municipal boundary shp. I then projected the shapefile to a different coordinate system and cleared the projection definition so ArcMap is putting it in the wrong place. We’ll pretend you’ve received this shapefile and nobody has told you its projection. You need it to line up with the zoning AutoCAD layer.
It is more likely you would be doing things the other way around, that is, that you would have received AutoCAD data with an unknown projection. However, spatial adjustment only works while inside an edit session. You cannot “start editing” AutoCAD data. You would need to convert it to a shapefile or feature class before you could spatially adjust it.
That brings us to step one. Start an edit session. Then, go to View –> Toolbars and check on the Spatial Adjustment toolbar if it is not visible already. Use the main dropdown menu to Set Adjust Data and chose your target layer (in this case the municipal boundary shape). You will have the option to adjust all the features or only the selected features. We will do all of them.
Then, press the button that looks just like Georeferencing’s “Add Control Points” to add your first Displacement Link. The lingo is different; the concept is the same.
Click on a distinguishing feature on the municipal boundary and then on that same feature in the zoning layer. Remember, you always start with the layer that you want to move and end by telling it where to go.
One difference that takes some getting used to is that there is no Shift or Fit to Display ability like there is in georeferencing. So, you can’t get the target layer and the reference layer into the same window before you start adding displacement links. You will have to keep zooming back and forth between them. If you can’t place your links as precisely as you’d like due to this, don’t worry, you can get them in the general vicinity and then press Modify Link to edit them.
Another difference between this and georeferencing is that placing links doesn’t automatically cause the layer to move. That only happens when you select Adjust from the dropdown menu. You will not be able to select Adjust until you have placed three links. I recommend doing so as soon as you can just to get things into the same universe. (There’s no limit to how many times you can re-adjust later.) Just like with georeferencing, place your three links as far apart from each other as possible.
Once you have adjusted, your links will disappear, and you will be able to refine your results by adding more links.
(After first adjustment)
At this point you may also want to experiment with different adjustment methods. The first three in the list are coordinate system transformations. You should stick with these when the only reason your data isn’t lining up is that you don’t know the projection.
- Affine (the default) will scale, skew, rotate, and translate the data.
- Similarity will scale, rotate, and translate but it will not skew. It will not change the aspect ratio or the shape of features– only their size, position and orientation.
- Project uses a more complex formula that is tailored to features that have been captured from aerial photography.
The last two should only be used if you know there are errors within your target layer that make it less accurate than your reference layer.
- Rubbersheeting behaves like it sounds– pulling and stretching your layer in a piecewise fashion. The areas that are closer to displacement links will move more dramatically. Identity links (the black and red square crosshair button) can be used to nail down features that should not move. If rubbersheeting is used it should be as a second step after transformation.
- Edge Matching is used to merge adjacent layers that have been split up. It makes sure features line up along one edge only.
You can preview the results of different adjustment methods in the preview window. You can also use the preview window to see the effects different displacement links will have.
Another way to check your work is by looking at the link table. If you are using a transformation adjustment method, get an idea of your RMS error. If you place a link that increases the RMS and Residual errors substantially, delete the link and try something else. I recommend keeping both the Preview Window and the Link Table open while you are working.
When you are happy with what you see, make a final adjustment and then save your edits.
Tags: coordinate system, spatial adjustment
Creating an XY Domain that covers your data extent
Problem: When loading data into into a feature class, you get the error “the coordinates or measures are out of bounds”
This issue most often arises for me when I am in ArcCatalog using the simple data loader tool. It can also arise when trying to import a feature class into a feature dataset, when creating new features during an edit session in ArcMap, or when performing spatial adjustment to move features to a new location. In each case, the error occurs when the extent of the feature class or feature database is not large enough to contain the new or moved features. The solution would appear to be “increase the extent,” however, it is not possible to change the extent once a feature class or feature dataset has been created. You need instead to create a new one with the correct extent. I will show you how to do that, returning to the example of the Oceanic layers I used in my post about preparing layers for merging with model builder.
Today, I would like to merge all my city point shapefiles into one feature class in my master Oceanic geodatabase. I have city points for Australia and Papua New Guinea. If I merely import the Australia shapefile as a feature class, and then attempt to load the Papua New Guinea points into that feature class, I will get this error message:
One of the points is out of bounds! By displaying the layers in ArcMap, it’s easy to see what happened. When I imported the Australia shapefile, the spatial domain was defined as the extent of that layer. The one Papua New Guinea point that didn’t get loaded fell too far east of that domain.
A good rule of thumb is: if you have a dataset that covers your entire domain, import that first and then load the other datasets into it. However, you often won’t. In that case you are going to have to take one or two extra steps.
Single step method (good enough for most purposes):
Create a new feature class. ArcCatalog will ask you to chose the projection of the feature class, and when you do, the extent will be set to the all of the mathematically possible values in that coordinate system. In other worlds, the whole world (and then some). ArcCatalog will then allow you to chose a schema for the new feature class. Import the schema from one of the layers you are going to load in.
You will now be able to load all of your layers into the new feature class without any out of bounds errors.
Two step method (better):
Add all of your layers to ArcMap. Then draw a box around all of the features and save it as a separate polygon shapefile.
This polygon shapefile will be used to precisely set your x,y domain. In order for this to work, the projection of the layer must be set to unknown. Clear the projection if it has one defined. Now, create a new feature class in ArcCatalog. When it asks you for the projection of your new feature class, import the projection of your polygon shapefile.
If this polygon has a defined projection, ArcCatalog will just do what it did before and set the extent as all mathematically possible values in that coordinate system. But if you import an undefined projection, ArcCatalog sets the domain as the spatial extent of that dataset. That’s exactly what you want. You’ll see an additional screen that shows you this is happening and even allows you to change the x,y domain values if you want.
Here, you can see what the domain looks like if it is defined automatically by the GCS WGS84 coordinate system.
And here is what is looks like when defined by my domain polygon with an unknown coordinate system.
Why is the second way better? It is preferable to have as small a domain as possible because a smaller domain allows for a greater resolution. Resolution is the minimum distance between coordinates. A higher resolution means that locations can be placed more precisely, because their coordinate positions are allowed to have more decimal places. Because data storage is finite, there will always be a trade off between extent and resolution. In most cases, the resolution gain will not be noticeable, especially if you are using a post-9.2 high precision geodatabase. But I put it here as a best practice.
Once you have loaded all your layers into your new feature class (error free!), define its coordinate system as your last step. This feature class is totally optimized for the data it’s holding.
Tags: coordinate system, data loader, domain, geodatabase
Defining the projection of AutoCAD DXF files
Problem: Your AutoCAD data isn’t lining up correctly with other layers in ArcMap
Most of the time, ArcMap “just works” invisibly behind the scenes to line up everything correctly. It doesn’t matter if the datasets are in different coordinate systems, and sometimes it doesn’t even matter if the coordinate systems are defined. This can make the times when something goes wrong seem like mysterious aberrations, hard to troubleshoot. But there really are only two reasons why it works and why it doesn’t.
Datasets will line up correctly if
- They are in the same coordinate system, even if one or more datasets is <undefined>
- They are in different coordinate systems, but all coordinate systems are correctly defined. ArcMap will do on-the-fly projection.
Datasets will not line up correctly if
- They are in different coordinate systems and one or more datasets are <undefined>
- One or more coordinate systems are incorrectly defined
If you are suffering from problem #1 or #2, the fix is running ArcToolbox’s “Define Projection” on any undefined or incorrectly defined datasets. That is, if you are working with typical ArcMap data like shapefiles and feature classes. The fix is different for AutoCAD data.
The City of Bloomington, Indiana, offers their GIS data in both shape and dxf format, so we can experiment. In this example, I have projected a road shapefile into GCS WGS84, but I have left the corresponding dxf road layer alone. The dxf is now in a different coordinate system than the shapefile, and that coordinate system is not defined. Hence, the two layers which should be exact duplicates are showing up worlds apart.
They will be brought back together if ArcMap knows what the coordinate system of the dxf is. But, as I said above, this shouldn’t be done like normal with ArcToolbox. Instead, it is better done in ArcCatalog. If you right click on the roads_city.dxf layer in ArcCatalog to bring up its Properties, you will see that the coordinate system is Unknown. Pressing the Edit… button will allow you to select a coordinate system. The metadata on the City of Bloomington website says this data is in NAD 1983 StatePlane Indiana West FIPS 1302.
Note: Be sure to edit the projection on roads_city.dxf itself and not the individual point, polyline, polygon components. The new projection will carry over into the entire collection.
Selecting this coordinate system will prompt you to save a .prj file. Be sure it is in the same directory with the same name as the dxf.
This is what is inside the .prj file if you look at it with Notepad:
PROJCS["NAD_1983_StatePlane_Indiana_West_FIPS_1302",GEOGCS["GCS_North_American_1983",
DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137.0,298.257222101]],
PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],
PARAMETER["False_Easting",900000.0],PARAMETER["False_Northing",250000.0],
PARAMETER["Central_Meridian",-87.08333333333333],PARAMETER["Scale_Factor",0.9999666666666667],
PARAMETER["Latitude_Of_Origin",37.5],UNIT["Meter",1.0]]
Note: Shapefiles have these too! If you delete them, the projection goes to Unknown. For some reason, ArcToolbox’s “Define Projection” tool creates these .prj files for shapefiles, but it doesn’t do so reliably for AutoCAD data. In some instances it only creates an .xml file for the AutoCAD data, which isn’t enough to tell ArcMap the projection. This problem appears to be fixed with 9.3, but 9.2 users will have more luck the ArcCatalog way.
All users will have better luck getting projection assignments to “take” if they make sure that:
- The dataset is not concurrently open in ArcMap
- The folder that holds the dataset is not set to “read-only”
- There are no spaces in the file name or path
Now, if you are successful, when you add the road dxf to an ArcMap session with your road shapefile, you will get a message asking you what geographic transformation to use. This means ArcMap has recognized the projection.
Pick a transformation and you’re good to go.
Now, what do you do if you don’t have metadata like the good City of Bloomington provides, so you don’t know the coordinate system of your CAD data? You have two options:
The easy way: Georeference it just like an image! This ability was added in ArcGIS 9.2. You can see below that I’m able to select the dxf in the georeferencing toolbar.
There’s some minor differences, so read here if you want the details or if you’ve never learned to georeference images:
- Transforming CAD datasets (ESRI Webhelp)
The hard way: If you’ve got an older version than 9.2, you will have to create a world file from coordinates at the corners. It’s an arduous process but it will work. Instructions here:
- HowTo: Transform CAD data to line up with other data (ESRI Technical Article)
I have done this successfully with CAD maps that have rectangle frames around them, making it easy to find the corners. I can see it being very difficult otherwise. If the world file will not work for you, there’s one other option.
The other hard way: Spatial Adjustment. In other words: “Georefrencing for vectors.” It works for shapefiles without attainable coordinate systems as well. I will be covering it in a later entry.
Tags: autocad, coordinate system
Calculating Polygon Areas
Problem: You need to get the areas of your polygons
I have often been asked questions like “What is the total acreage of forest in our study area?” or, “Has the total forested area increased or decreased since we last mapped it two years ago?” The answer starts with being able to calculate polygon areas. After that it’s just addition and subtraction.
I used to have to use VBA scripting to calculate polygon areas. Now the task has been simplified with the Calculate Geometry tool introduced in ArcMap 9.2. However, there are some things you need to know to get good results with this tool.
There’s a lot of little parks around the San Francisco bay area in California that will serve as a good test case.
Let’s add the area of each park to the attribute table. The first step is to create a new field to hold the values. Chose field type “Double” (Double-precision floating-point number) which allows lots of decimal places. Then, right click on the field name and chose “Calculate Geometry…” from the context menu. You will then get to choose your units. In this case, decimal degrees are the only option because the layer is in a geographic coordinate system. Decimal degrees are pretty useless.
Fortunately, the tool allows you to use the coordinate system of the data frame, so you don’t have to worry about re-projecting. You just need to change the data frame into a projected coordinate system, one that uses a linear unit of measure, such as feet or meters. UTM (Universal Transverse Mercator) is widely used. This image from wikipedia shows the San Francisco bay area is in UTM zone 10N.
To change the projection of the data frame, select Data Frame Properties from the View menu. On the Coordinate System tab, pick from the Predefined – Projected Coordinate Systems folder.
Now, the Calculate Geometry tool gives you choices like square feet, square miles and acres. I picked acres, a good tiny park sized unit. I got results back with 6 decimal digits, which sure seems good.
But wait. We used a common projection which is great for most purposes, but not particularly for this one. UTM preserves shape and direction, which makes things look right on maps. However, it distorts area and distance. Exactly not what is needed here. The distortions may be small, but if you want to be as accurate as possible (and what GIS Analyst doesn’t?) it is better to use an equal-area projection. Albers projections preserve area while distorting shape and distance. If you root around in the list, you’ll notice there’s several Albers equal-area projections available for different regions, such as Albers North America and Albers USA Contiguous. It is best to pick the most local one you can find. In the State Systems folder, there’s a California (Teale) Albers. Perfect!
Changing the data frame to this, and recalculating the area in a second field, does yield different, better results.
If you need to get the total area of all the parks, don’t dig out your calculator. Just right click the field name to run the Statistics tool.
Tags: area, coordinate system, polygon