Using the command line to convert between metadata formats
Problem: You have a lengthy metadata record that you want to preserve, but it has been created using a different stylesheet than your standard.
Just a few days before I was going to take a course in python scripting, I noticed the command line button on the ESRI toolbars.
It is there in ArcMap and ArcCatalog. I wonder why I’d never noticed it before. Turns out you can do a lot of cool things with it. The portion of every Help document labeled “Command line syntax” is talking about things you can type into the command window. Plus, I learned in my class later that with ArcGIS 10, it will be renamed to the Python Window because we will be able to run python scripts directly from it! But I am getting ahead of myself.
There are certain things you can only do from the command window, and today I will talk about one of them: converting metadata from one format to another. The two prominent metadata standards are FGDC: Federal Geographic Data Committee and ISO: International Organization for Standardization 19115.
If you want to use data from an organization that follows a different standard than you do, you might need to do a conversion. For example, the metadata for World Wildlife Fund’s Global Lakes and Wetlands Database is in FGDC format. In order to view it in ArcCatalog, I need to have an FGDC stylesheet selected.
With the right stylesheet, all of the pertinent information is displayed in the metadata window.
If I switch to an ISO stylesheet, what I see is not nearly as helpful.
You could consign yourself to switching between stylesheets, but I suspect many people work in places that insist all of the metadata for their collection of geospatial records be in the same format. If you need to do a conversion, there is a faster way than typing all of the information back into the metadata creation wizard.
ESRI has provided a set of four metadata translators with the ArcGIS desktop install. They should be located in the \Program Files\ArcGIS\Metadata\Translator folder. You can run these translators from the command line window. First, pick the one you need.
- FGDC to ISO 19139 (FGDC2ISO19139.xml)
- ESRI-ISO to ISO 19139 (ESRI_ISO2ISO19139.xml)
- FGDC to ESRI-ISO (FGDC2ESRI_ISO.xml)
- ISO 19139 to ESRI-ISO (ISO19139_2ESRI_ISO.xml)
Then press the command line button, and type ESRITranslator into the window. If you press the spacebar, a tool tip will pop up that tells you the arguments you need to enter next. In this case it will say
ESRITranslator <source> <translator> {output} {logfile}
Input within carrots <> is required; input within brackets {} is optional. For me, <source> is the path to the Global Lakes and Wetlands metadata .xml file. <translator> is the path to the ESRI Translator I want to use ( FGDC to ESRI-ISO ). And {output} is the path to where I want to store the new converted .xml file. I’m not going to specify a logfile.
My code looks like this:
(click on image to see full size)
Notice a couple things here. My cursor is in the path to the translator file, and <translator> is bolded in the tool tip. This is intentional. If I move the cursor to other arguments, the bolding will change accordingly. Use this to make sure your syntax is correct.
Also notice that I have surrounded the path to the translator file in quotes. Quotes are optional, but you must use them if you have spaces in your path name. Otherwise the space will be misinterpreted as the beginning of the next argument. Going through and checking what is bolded where will help to catch this sort of thing.
Press enter to run the code. Then, press the Import Metadata button and browse to your new .xml file.
Now it looks how ISO should!
Tags: command line, fgdc, iso, metadata, xml
Doing network analysis
This is post #3 in a three part series. It is meant to be read after Creating a Network Dataset.
Now that we’ve done all the hard work building our network dataset, it is time to have some fun putting it to work. Most presentations would cover this part first to lure you in. I might have done things backwards. If I haven’t scared you away yet, and you’ve brought along a working network dataset, let’s explore what Network Analyst can do with it.
I will give examples of the first three Network Analyst tools: Route, Service Area and Closest Facility. Each of these revolve around facility locations paired with the network dataset. The City of Portland GIS Catalog doesn’t have a Points of Interest or Buildings layer, so I used ESRI Streetmap USA data for the point locations.
Route Tool
The Route solver finds the best path between any number of stops. Let’s pretend we are tourists who want to see a bunch of sites downtown and need some help planning our day so we’ll spend the least amount of time driving. I opened up ArcMap and added my new network dataset and the ESRI Recreational Areas point layer. Zooming to the Portland area shows 12 sites on both sides of the Willamette River.
Check on the Network Analyst toolbar, and then select New Route from the drop down menu. Also, click the button right next to the drop down menu to show the Network Analyst Window. You’ll be using it a lot.
The window will have a list of Stops, Routes and Barriers, all empty for now. The first step is to populate your list of stops. You can create stops manually by activating the Create Network Location Tool and clicking on the screen while your pointer is in the flag with crosshairs shape. Or, you can load them from another layer. That’s what we’re going to do. Select the 12 sites that you see on screen. (This step is important because you don’t want to be working with all the recreational areas in the entire USA. I skipped it the first time and proceeded to crash my computer). Once you’ve selected your subset, right click on Stops and select Load Locations…
It’ll grab their name from a name field if one exists, and nicely number them for you. Now click the Solve button. It’s generated a topsy-turvey looking route, due in part I’m sure to all the one-way streets downtown.
If you click the driving directions button, you’ll see the route has 80 steps and takes about 14 minutes.
We can probably do better than this. There’s an obscure button that will allow you to change some of the parameters used to calculate your route. It’s hidden on the top right of the Network Analyst Window.
The Analysis Settings tab has a Reorder Stops to Find Optimal Route box that is checked off by default. This means that if you added the stops manually, it will visit them in the order you created them, which might make sense. But if you loaded them from a file like we did, it will visit them in the order of random numbers it assigned, which definitely doesn’t make sense. Check the Reorder Stops box on, and check off the two boxes under it: Preserve First and Last Stop.
Now, the software will have the freedom to plan an itinerary that maximizes efficiency. Let’s see what difference it makes. After you’ve pressed OK to dismiss the Route Properties box, press the Solve button again.
That’s much better! It now wants me to visit the Children’s Museum first instead of last. And it has rearranged the entire middle so it’s not a mess. This new route saves us 5 steps, 3 miles and 4 minutes.
Service Area Tool
Now let’s pretend we work for Woodland Park Hospital and we want to know the area around it than an ambulance can reach within 5 minutes. We would select New Service Area from the drop down menu, and then load that one hospital point into the Facilities list.
I personally like to go into the Service Area Properties box and change the Polygon Type to Detailed (under the Polygon Analysis tab) because, why wouldn’t you? It’s set to calculate for 5 minutes by default, but you can change this to anything by editing the value in the Default Breaks box (under the Analysis Settings tab). If you want to solve for multiple times, separate them by commas. Here’s how it looks if I solve for 1, 3, and 5 minutes with high detail:
By the way, if you want to change these colors you can edit the symbology just like you would any other layer in the table of contents. And, if you wanted to, you could turn off the OneWay restriction because ambulances don’t need to obey it.
Closest Facility Tool
For the final scenario we’ll pretend to be home owners in the southwest portion of the city. We are planning a family and would like to determine the closest school to our house. The first step to solve this problem is to select New Closest Facility from the Network Analyst drop down menu, then press the Create Network Location tool button, and click on the map at the location of the house.
The new location will automatically be added to the Facilities list, but you want it in the Incidents list. Either drag it down there, or make sure the Incidents list is highlighted before you use the Create Network Location tool. The point will be automatically named Graphic Pick 1. You can double click on its entry in the Incidents list to rename it.
I purposefully picked a location that seems equally close to three different schools. The tool will tell me which is actually the fastest to drive to, not the closest as the crow flies. I selected a bunch of schools around it (more than just those three, to be safe) and loaded them into the Facilities list. Then I pressed the Solve button. The result is that Markam School is the closest at 0.7 miles away.
The tool shows you the route to get there, too. You can open up the Directions Window to see driving directions as well.
These are pretty handy, aren’t they? I’m in the mood to plan a vacation now just so I can play with the Route Tool some more!
Tags: driving directions, network analyst, service area
Creating a network dataset
This is post #2 in a three part series. It is meant to be read after Preparing your Data to Become a Network Dataset.
A network dataset can be created from a shapefile or a feature class. If you are creating one from a feature class, it needs to be in its own feature dataset. Make sure you have the Network Analyst extension checked on, then right click on your streetline shapefile or the feature dataset that holds your streetline feature class, and choose “New… Network Dataset”.
After you give it a name, you will have the option the set Connectivity. Select the connectivity policy that you decided on before.
If you have designated z-levels the way I described, the next screen you encounter will automatically have these elevation settings. Keep them.
I haven’t tackled turn modeling yet, so select “No” on the next screen unless you’ve figured that out. (In which case, do you want to do a guest post?) 
Now you are at the Attributes screen. If you’ve designated one-way streets the way I described, they will already be set up as a Oneway Restriction attribute. In addition, you’ll have a RoadClass Descriptor Attribute. Network Analyst uses this attribute to make the wording for turn-by-directions read correctly. There are 5 categories:
1 Local Street
2 Highway
3 Ramp
4 Ferry
5 Roundabout
It is automatically configured to try to inspect your attribute table and pull out these categories, but there’s no guarantee it’s going to work right. You should look at the code to check. Select it and click on the Evaluators button. Then select the first one in the list and click on the Evaluator Properties button.
Here’s what appears in my Pre-Logic VB Script Code box:
rc = 1 'Local road
Select Case UCase([CFCC])
Case "A10", "A11", "A12", "A13", "A14", "A15", "A16", "A17", "A18", "A19"
rc = 2 'Highway
Case "A60", "A63"
rc = 3 'Ramp
Case "A65", "A66", "A68", "A69"
rc = 4 'Ferry
Case "A62"
rc = 5 'Roundabout
End Select
It is picking attributes out of my Census Feature Class Code (CFCC) column. But is it not classifying them in the same way my metadata says they should be. I have some evidence my metadata is out of date, so maybe the code knows better, but I can’t be sure. So, I replaced the code with this, which I am sure about:
If [FTYPE] = "FWY" Or [FTYPE] = "HWY" Then
rc = 2 'Highway
ElseIf [FTYPE] = "RAMP" Then
rc = 3 'Ramp
Else
rc = 1 'Local road
End If
I didn’t have anything that was obviously ferries or roundabouts. Make sure rc, or whatever variable you use, is in the “Value =” box below the code window. Make the same changes for the To-From direction as well.
Now it is time to create your Cost Attributes. Press the Add… button and create a Distance Attribute. Make sure the Usage Type is Cost and that you set the units and data type to match your length field. Uncheck the Use by Default box unless you don’t have information to create a Time attribute.
It will have a yellow exclamation point beside it until you press the Evaluators button and define how it should be calculated. Set the Type to Field and the Value to your length field for both directions.
Follow the same procedure to create a Time attribute, except this time set the units to Minutes and keep Use by Default checked on. When you get to the Evaluators screen, set the Type to Field and then click the Evaluator Properties button so you can enter an expression (unless there is already a time field in your attribute table). I used the expression ( [LENGTH] / [Speed] ) * 60 in the “Value =” box. This works if you have a field for length in miles, speed in miles per hour, and want time in minutes. Adjust accordingly.
Next create a Hierarchy attribute (Usage Type: Hierarchy). You will probably want to leave the Use by Default box checked, but I will leave that up to you. (These default settings can still be changed when you run scenarios later). If you created a new field for it, then calculate it using that. Otherwise you’ll have to code a classification expression like I did for the RoadClass attribute… only this time with categories 1-3.
Finally, if you thought of any potential restrictions to travel earlier, create Restriction attributes for them now. Create a different attribute for each type of Restriction. I set one: “Private”, for private roads and driveways, so I could avoid them if need be (like if I was driving a commercial vehicle). I didn’t use it by default because most of the time this won’t be an issue. Restriction attributes are based on a Boolean value (True or False) so my classification expression looks a little different:
Here’s my list of attributes in the end:
If you’ve done everything right thus far, you’ll have the option to configure driving directions. (It didn’t let me the first time around, I think because I didn’t create a Distance attribute). These are cool and you definitely want them! Click the Directions… button to see/change the settings. It’s pretty smart and gets everything right for me automatically. You can click anywhere to bring up drop-down menus and change things.
I didn’t have anything in my attribute table that helps with labeling highway shields (like route numbers). But I do have a field that says which city/county each road segment is in. So I pointed to that as the boundary field on the Boundary tab. Now, the driving directions will tell me if I cross over from one jurisdiction to another.
You’ll get a summary at the end, and then it’s time to click FINISH! Go ahead and Build when it prompts you. Then, add your new network dataset to ArcMap because it is time to play.
Stay tuned for installment #3: Doing network analysis
Tags: driving directions, junction, network analyst, VBA
Preparing street data to become a network dataset
Problem: You are unfamiliar with the Network Analyst extension and you would like to maximize the potential of your streetline data.
Recently I encountered my first network dataset. This is a special file created by, and meant to work with, the Network Analyst extension.
Route planning is the first thing that pops into my mind when I think about what a GIS does, yet I’d never even turned on that extension before. This is going to be a very basic introduction for people who might be in the same boat: interested in what Network Analyst can do, but don’t know where to start.
I am going to post it in three installments
1. Preparing your data to become a network dataset
2. Creating the network dataset
3. Doing network analysis
The first thing you will need is a streetline layer. The more information it has in it, the better. It should at least have road names, road types, and direction of travel (for one-way streets) designated, or its use will be very limited. You can use the ESRI StreetMap data if you have access to it, which is optimized for use with Network Analyst and even comes with a pre-made network dataset. If you’re going to do that you can skip ahead to installment #3. Otherwise, read on:
Preparing your data to become a network dataset
I downloaded a streetline shapefile from the City of Portland GIS Data Catalog.
The fundamental bit of information network analyst needs in order to work is a Cost Attribute. Cost is either the travel time or distance on each road segment. It is the thing the model is trying to minimize when calculating the best route. Distance is easy to add to any dataset. Create a field (type: double) if one is not there already, and then use the Calculate Geometry tool. Chose miles or kilometers, whichever you prefer, as your length unit. Note: your dataset needs to be in a projected coordinate system for this to work.
If you know, or have a basis upon which to estimate, the speed of travel on each road segment, you can also create a Cost Attribute based on time. This will make your results much more meaningful. My dataset designated road types, which allowed me to make an educated guess about speed limits. I added a Speed field and populated it with my miles per hour estimate. The Insurance Institute for Highway Safety has a table of maximum speed limits in each state. Oregon’s is 55 for urban interstates. So, I used 55 for highways and freeways, 45 for major/primary roads, 35 for secondary roads, 25 for minor/local roads and ramps, and 15 for driveways.
These speed and distance fields in the attribute table will be used to calculate a time attribute later in the process. Moving on:
The next fundamental attribute to have is Hierarchy. Network analyst uses it to create more realistic routing based on the fact that people prefer to travel on larger roads. It is an integer value that ranges from 1 (major) to 3 (minor) based on road class / road level. If your roads are already labeled as primary, secondary, and tertiary, you’re set; otherwise, you’ll have to make some classification decisions. You can add a new integer field to the attribute table for it (that’s what I did). Or, you can calculate it later, like I’ll explain how to do with the time attribute.
Next, you will need to flag one-way roads with their direction of travel. I struggled with this for quite awhile before stumbling upon a procedure that works correctly, so I recommend doing it just this way:
- Create a new string field in the attribute Table called OneWay.
- Change the symbol of your roads to a cartographic line with the arrow pointing right. This will allow you to see what direction the line was digitized in.
- Bring up a Google map of your area, or something else that allows you to see the direction of travel.
- Put an “F” in the OneWay column for any road that has been digitized along with the direction of travel.
- Put a “T” in the OneWay column for any road that has been digitized against the direction of travel.
- Use an “N” for any roads that don’t permit travel in any direction.
- Use any other value for 2-way streets. (I used “X”).
(click on image to see full size)
When you create your network dataset later, Network Analyst will recognize this formatting and will know what to do with it.
Finally, examine the way your road lines were digitized, especially around overpasses and tunnels. If they have a unique segment between each intersection (that is, they break anywhere a person could theoretically turn), then you can use End Point connectivity—the default. If, however, your roads are long lines that don’t break when they cross, you will have to use Any Vertex connectivity. The type of connectivity you chose tells Network Analyst where to create Junctions. Junctions are a special point layer that will be generated along with your network dataset. They are created wherever a person could transfer from road to another.
Regardless of what connectivity you chose, you will need a way to designate overpasses and tunnels so that people will not be routed to turn onto roads that are inaccessible because they pass below or above them. This is done using z-levels. Create two new integer fields to hold your z-level information. Call one “F_ZLEV” and the other “T_ZLEV”. The field prefixes stand for “From” and “To,” respectively. First, designate all ground level roads as z-level 0 in both fields. Then designate overpasses as 1 and tunnels as -1 in both fields. (Hopefully, your attribute table will tell you where these are. Mine had a “Structure Type” field). Now, symbolize with a cartographic line again as I described in Step #2, so you can see the digitization direction. Change F_ZLEV to 0 at the beginning of raised or lowered segments where they meet level road. Change T_ZLEV to 0 at the end of raised or lowered segments where they meet level road. The idea is that a route will only go through a junction where the z-levels match.
(click on image to see full size)
You are almost ready to move on to the next step, but before you do, think about any other potential restrictions to travel. Do you have roads that are marked as “under construction,” toll roads or private roads? Make a note of how they are designated in the attribute table so you can flag them later.
Stay tuned for installment #2: Creating a network dataset
Tags: cost attribute, network analyst, one way street, z value
Making a map look prettier
Problem: Your map is going to be used for a special purpose and it needs a little something extra to give it that polished, professional look.
On St. Patrick’s Day, the ESRI Mapping Center blog posted one of the award winners of the 2009 International User Conference Map Gallery Competition. The title of the map is “Geo-Geneology of Irish Surnames.” Besides being interested in the subject matter because I am of Irish descent, I think it is a beautiful presentation. So beautiful, in fact, that I printed out two copies on photo paper: one for my office and another as a Saturday party decoration.
I started thinking about the ingredients that make this map look so good. It was made entirely within ArcMap, which means it’s possible take ideas from it that would improve the appearance of any map created on the job. I noticed four things in particular that I have seen on other professional maps hanging on walls, but which I hadn’t tried before. I taught myself how to do them, and now I will teach you.
1. Shaded Relief Background
The map uses a color gradient to display different elevation values. This imparts a subtle texture that makes the background spring to life. I was able to achieve a similar effect using a combination of a hillshade with a carefully symbolized DEM surface. I used the USGS GTOPO30 for this. GTOPO30 is a Global Digital Elevation Model with a horizontal grid spacing of 30 arc seconds–approximately 1 kilometer.
First, I clipped it to my map area in order to make it easier to work with. ArcToolbox’s Data Mangement Tools –> Raster –> Clip function will clip any raster to bounding box coordinates that you either type in directly, or import from a shapefile. Here, I’ve imported the bounding coordinates from a shapefile of the Ireland country boundary.
Now that the DEM has been clipped to a manageable size, creating a hillshade will go much faster. Hillshading is a method of representing relief on a map by depicting the shadows that would be cast by high ground if light were shining from a certain direction. You can create hillshades from the the 3D Analyst toolbar.
When the tool comes up, it will ask for several values which are fine to keep at the defaults, except for the Z-factor. The ESRI Help document on hillshades notes that if the x,y coordinates of your data are in angular units (degrees), the hillshade output is going to look funny. This happens because the elevation values are in linear units, and the length of a degree of longitude changes with latitude. So, if your x,y units are in decimal degrees and your z units are in meters (which is the most common case and the case with GTOPO30, which is in GCS WGS 1984), ESRI recommends using the following table of z-factors:
| Latitude | Z-factor |
| 0 | 0.00000898 |
| 10 | 0.00000912 |
| 20 | 0.00000956 |
| 30 | 0.00001036 |
| 40 | 0.00001171 |
| 50 | 0.00001395 |
| 60 | 0.00001792 |
| 70 | 0.00002619 |
| 80 | 0.00005156 |
Ireland is at about 50 degrees latitude.
The output, which I named “shadow”, is a grayscale shaded relief that looks like the land surface with hills and valleys. We are going to drape it over a color-coded DEM. The hillshade will provide the light-dark values and the DEM will provide the hue values. I found that my hillshade looked the best with the following contrast and transparency percentages. (You can find these settings on the Layer Properties, Display tab).
Now, go back and choose colors for the original clipped DEM (gtopo_1km_Clip1). After a lot of trial and error I got my best results with 6 color classes, ranging from green at the lowest elevations through brown to white at the highest elevations.
2. Gradient Water Bodies
Using a gradient fill to symbolize the lakes creates an effect called “sunglint”. I’d never heard of it before, but it’s easy to to do, and quite pretty. In the Symbol Property Editor, change the Type to Gradient Fill Symbol. Raise the Interval value to something higher than the default 5. This will blend the colors better. You’ll stop noticing the difference much over 50. Now, pick a blue color ramp from the list.
If you want to, you can tweak the pre-made ramps by right-clicking on them and selecting properties. I didn’t like the darker blue end of mine, so I removed it.
Note: If you click properties again, you can change the two colors that make up the ramp, as well as the blend algorithm and the black and white brightness.
3. Coastal Vignette
The Irish map makes dramatic use of this, so that the coastline appears to glow out of the–in their case–green background. Most other maps use a subtler effect of the coast fading into blue water. To create that look, first change your background color by going to the Data Frame Properties, Frame tab.
Then, create multiple copies of your country boundary in your table of contents. Each copy should be symbolized with a white hollow border that has increasing width and transparency as you go down. Here are the settings I used:
4. Colorful Administrative Boundaries
This takes a bit of time, but the result is lovely. The idea is to have each county outlined by a border that’s shaded a different color from the counties surrounding it. The borders need to touch, but not overlap. And, the borders need to fade, like the coastline, but not quite as much. So, how to do it?
Before you start picking colors, you need to solve the overlap problem. (I learned the hard way that this step will undo your color choices). Symbolize by category: province name, then right click on any province in the list and select Properties for All Symbols. Click the Properties button, the Outline button, and then the Properties button once again. Now, change the line Type to Cartographic Line and the Offset to -0.5.
Note: The offest should be negative half the width of your line. My line width is 1.
Now, go back to the first screen and pick your border colors. If you keep the window open and keep hitting Apply, you can check your results as you go to make sure you don’t put two of the same color next to each other.
When you’re finished, make a second copy of this layer, and put it underneath the first in the table of contents. Then, double the border width and the offset. Important: also go to the Cartographic Line tab and set the Line Caps and Line Joints to Round. This will avoid spikes from showing up at wider widths. Fair warning: when you do this, your colors will be all be reset to black, and you will have to go back and fix them one by one. I’m sorry.
It’s totally worth it. Set the transparency on the bottom layer to 50%. And, you end up with:
Isn’t that nice?
Tags: cartographic line, clip, color ramp, hillshade, symbol, vignette
Creating a list of keywords from the attribute table
Problem: You need to summarize the contents of a dataset without wasting time typing a long list.
Natural Earth is a crisp basemap dataset available for the world at 1:10m, 1:50m, and 1:110m scales. I downloaded one of their most detailed layers, the 1:10m Admin 1 – States, Provinces shapefile for use as my example.
If you were asked to write a metadata record or some other type of report about this dataset, you would want your description to be as complete as possible. You might find it necessary to generate a long list of place or topic keywords. Often, the attribute table already includes this information. You just need a good way to extract it.
For instance, if I symbolize by category on the ENGTYPE_1 field of my natural earth layer, I get a long list of administrative area types. There’s way more than just states and provinces here! If only I could select this list and paste it somewhere else to have in a usable format. Unfortunately, it’s not that easy.
The solution I have come up with is not perfect. I keep thinking there must be a better way. If any of my readers know of one, I would love the feedback. For now, this method certainly beats having to type the entire list:
First, open attribute table and right click on the field you want to turn in to a keyword list. Then, select Summarize…
Chose one summary statistic field. It doesn’t matter which, but works a little better if you chose something that’s always going to be the same. I chose “CheckMe” because that’s basically a flag (yes or no) field. It doesn’t matter which summary statistic (minimum, maximum, first, last etc.) you pick either. You won’t be using any of this information–it’s just a way to get ArcMap to make a list for you that includes each category one time.
Hit the browse button in the Specify output table section. Then, change Save as type to Text File.
Open up your text file in Notepad and remove the first line (the field names). Then, do Edit — > Replace and replace all commas, quotation marks, numerals 1-9 and whatever text phrase was in your summary statistic field with nothing.
When you are done, you end up with this. If you want to add commas and move these all into one row, it’s pretty easy from here.
Tags: attribute table, keyword, list, report, summarize, text file
Creating vertical profile graphs
Problem: You have some contour lines and want to chart elevation changes along a path
We got a 3D view of a neighborhood in my previous entry about displaying data in ArcScene. As I was panning around I got to thinking about what it would be like to sled down some of those hills. This one cul-de-sac looked particularly nice.
Hikers sometimes use topo maps to make vertical profile graphs along prospective trails so they know what they are getting into. Engineers use them when they are working on the grade of the new road. Today, we’re sledding enthusiasts wanting to see how much fun we’re going to have come winter.
In order to create a profile graph down the hill I spotted, I will need to move all my layers back into ArcMap. I’ll need the TIN surface to provide the height information and the roads so I know where to draw my line.
Once you’ve brought the layers up, go to Tools –> Extensions… and make sure you have the 3D Analyst extension checked on. Then go to View –> Toolbars to turn on the 3D Analyst toolbar. 3D Analyst needs a raster elevation surface to work with. If you are starting with vector topo lines, you’ll need to create a TIN (Triangulated Irregular Network).
As I explained in my ArcScene entry:
To do that, go to the 3D Analyst drop down menu, and select Create/Modify TIN –> Create TIN from Features. You just need to tell it which feature layer to use to create the TIN, which attribute has the height information, and which triangulation method to use. Soft Line is best for topos.
Make your TIN the target layer. Your screen will look something like this.
Now, press the Interpolate Line button that I moused-over above. Then, draw a line along the middle of the road from the point where the imaginary sled will start to the point it will stop. The line will show up as a graphic element. Make sure it is selected (it should be by default) and then click the Create Profile Graph button. Here’s what mine looks like:
Hmm, not really the even downward plunge I was hoping for. Good thing I checked! If you right click on the graph and select Properties… you can change the titles to make them more descriptive.
I edited my main title and bottom + left axis labels to get this.
Note: You can change the appearance of the profile line on the Series tab, and you can change the appearance of the entire chart under Advanced Properties.
I have found it a good idea to create a vector line over the top of the graphic element profile line. That way I’ll have it saved in a format that I can bring into other maps or, for instance, bring back into my ArcScene view.
Tags: 3D, chart, TIN, topography, vertical profile
Embedding a custom Google mashup
Problem: You need to display GIS data on a webpage on top of a familiar basemap with basic zoom and pan tools.
There are many ways to do this. The one I will explain today doesn’t require programming skills, specially configured servers or proprietary software. Note: I borrowed heavily from Havard University’s Google Maps tutorial for this entry. They have an excellent free web-based GIS Technical Training Workshop Series.
The basic map they provide is all you really need to get started. The HTML file is structured and commented in such a user-friendly way that you will be able to start editing it to make common customizations to your map almost immediately. No need to wade through the Google Maps API Documentation until you’re ready to try something more complicated.
Go to http://maps.cga.harvard.edu/samples/basic.html and save the html source document (basic.html) to your harddrive. Then open it in a text editor.
The first thing you will probably want to change is the area of the world that the map is showing. To do that, change this bit of code, which is right now centered and zoomed to show the United States.
//Sets the initial map location and zoom level (latitude, longitude) in decimal degree format (zoom level) 1 - 21
gmap.setCenter(new GLatLng(39.10960, -96.5), 4);
For my example, I’m going to make a map of the recent Haiti earthquake. There’s been a lot of that going on recently as geo-Pros have been banding together to speed information to the relief workers.
I’ve changed my center coordinates to the location of the main quake, and upped my zoom so Hispaniola island fills the screen:
gmap.setCenter(new GLatLng(18.44, -72.54), 7);
Now, I’m going to change the starting style to make it a little less busy, since I’m going to be adding things. Uncomment the second line in the code here.
//Sets the initial map to load. Change between PHYSICAL, SATELLITE, HYBRID, NORMAL. Default is NORMAL
//gmap.setMapType(G_PHYSICAL_MAP);
When commented out, it will default to NORMAL. If you want to change that, you will need to delete the // in front and change it to one of the options above. Like so:
gmap.setMapType(G_HYBRID_MAP);
Note: PHYSICAL is an attractive terrain map that is not available in standard google maps. It’s tempting to use it, but it’s still too bright for the things I’m going to add next to show up well.
My map is centered on the location of the quake, but I also want to add a pointer to show exactly where it happened. To do that, insert the following code after the line where you changed the background map style.
//Adds a marker with window information
function createMarker(point,html) {
var marker = new GMarker(point);
GEvent.addListener(marker, "click", function() {
marker.openInfoWindowHtml(html);
});
return marker;
}
var point = new GLatLng(18.44,-72.54);
var marker = createMarker(point,'Magnitude 7.0 Earthquake Jan 12, 2010');
gmap.addOverlay(marker);
//End of add marker code
The editable areas in red are where you can change the coordinates of the marker and the description in the pop-up box.
I will also plot the locations of a couple aftershocks that occurred over a week after the initial event. I got these locations from the USGS Earthquake Search page at http://neic.usgs.gov/neis/epic/epic_circ.html
var point = new GLatLng(18.38,-72.85);
var marker = createMarker(point,'Magnitude 4.9 Aftershock Jan 21, 2010.');
gmap.addOverlay(marker);
var point = new GLatLng(18.48,-72.45);
var marker = createMarker(point,'Magnitude 4.4 Aftershock Jan 22, 2010.');
gmap.addOverlay(marker);
You can continue adding as many points as you like in this fashion.
You will probably want to change your markers so they don’t all look the same. In my case, I would like to distinguish the earthquake from the aftershocks. You can use any of the icons shown on this page, instead of the default marker. To do that, add this code block right before the //Adds a marker with window information block:
//javascript that sets up variables that enable the map to draw a custom icon.
var baseIcon = new GIcon();
baseIcon.iconSize=new GSize(32,32);
baseIcon.shadowSize=new GSize(56,32);
baseIcon.iconAnchor=new GPoint(16,32);
baseIcon.infoWindowAnchor=new GPoint(16,0);
var custom_icon = new GIcon(baseIcon, "http://maps.google.com/mapfiles/kml/pal3/icon28.png", null, "http://maps.google.com/mapfiles/kml/pal3/icon28s.png");
//end of custom icon code
Change the red pal group number and icon number to match the icon you want to use. Then, add the blue text to the end of these lines:
//Adds a marker with window information
function createMarker(point,html,icon) {
var marker = new GMarker(point,icon);
GEvent.addListener(marker, "click", function() {
marker.openInfoWindowHtml(html);
});
return marker;
}
var point = new GLatLng(18.44,-72.54);
var marker = createMarker(point,'Magnitude 7.0 Earthquake Jan 12, 2010.', custom_icon);
gmap.addOverlay(marker);
Note: if you wanted to use two types of icons, you could define (for example) a custom_icon2 above and assign it below, in the same manner.
It is simple to add KML files, which were developed for use with Google Earth, to these maps. Although shapefile is still the most common format, KML is becoming more and more available. My favorite new data search tool, the Geocommons Finder! offers its entire catalog in KML format (along with SHP and CSV).
To add a KML layer that shows the locations of hospitals in Haiti, you’d insert the following code right before the //Close bracket for the function initialize:
//Adds a kml overlay
var geoXml = new GGeoXml("http://finder.geocommons.com/overlays/20906.kml");
gmap.addOverlay(geoXml);
//End of kml overlay code
You can add as many of these as you like, and change the URL to point to whatever file you’d like. Note: If you, like me, have most of your data in other formats, you can use Havard’s Export to KML: Version 2.4 ArcGIS extension (see list on right side of page) before adding it to a mashup.
Here’s my Haiti map with all the changes I described. View the source to see them in context.
Tags: Google Maps API, javascript, kml, mashup, symbol
Displaying in 3D with ArcScene
Problem: You have some elevation or multipatch data that you would like to visualize in three dimensions.
For several of my entries I have been using layers from the robust catalog over at DC GIS. They offer pretty much everything needed to make a map of the District. In addition, some of the data layers contain elevation information, such as the 1 meter topo lines and the mulipatch buildings.
Note: Multipatch is a unique geometry type that can contain 3D models with vertical and overhanging faces.
These layers lend themselves to being rendered in 3D, and ArcScene has been formulated for that purpose.
ArcScene will show up in your Start Menu along with ArcMap and ArcCatalog. There is also a button that launches it from the 3D Analyst toolbar.
You can add data to ArcScene using the Add Data button just like you would to ArcMap, but it will look very different. Any of the height information embedded in the layers will automatically be applied to form a 3D world.
Here’s how the multipatch buildings, topo lines, road areas, lot boundaries, tree points and swimming pools from DC GIS look when I first add them.
It’s pretty neat, but needs some work. Some of layers, like the lots and roads, appear to be floating beneath the ground. This is because those layers don’t have height information, so ArcScene doesn’t know where to put them. In order to proceed, we need to make a surface from those topo lines. To do that, go to the 3D Analyst drop down menu, and select Create/Modify TIN –> Create TIN from Features. This will create a Triangulated Irregular Network to act as an opaque ground surface that the other layers can be draped over.
You just need to tell it which feature layer to use to create the TIN, which attribute has the height information, and which triangulation method to use. Soft Line is best for topos.
Now I have a solid ground surface with realistic shadows to enhance the 3D effect.
In order to raise up the other layers so they sit on top of the ground, right click on them to open their Properties. Then go to the Base Heights tab and tell ArcScene to obtain their heights from the surface you just created.
I did this for all my layers and also changed the symbols to some more pleasing colors. Now it’s looking like a real neighborhood!
Those multipatch buildings are by far the coolest part. They’re likely pretty rare in the typical GIS catalog, however. If you’re like most people and just have normal (boring) building footprint polygons, at this point your scene would look thusly:
You can add more realism by “extruding” the buildings up from the ground by a uniform height. Go back to the Layer Properties and click on the Extrusion tab.
Play around with different values until it looks right. I found that 8 was about the height of most of the multipatches. It just looks like they’ve all got flat roofs now.
If I wanted to, I could extrude the roads and swimming pools just a bit too. I tried it, and decided they looked better flat. I also changed the tree symbol to Big Leaf Maple (available through More Symbols –> 3D Trees), checked off the Edge Types symbol on the TIN, and turned on the lot boundaries as hollow polys. Just some ideas to show you different things you can do.
Whatever you do, you are going to want to try a fly-through. You won’t be able to resist. I sure couldn’t.
Press the button that looks like a little bird, then click in the scene where you want the bird to go, and use the mouse to soar through the streets at eye level. If you are giving a presentation, this is a guaranteed crowd pleaser. Here’s some “photos” I took while flying through my scene.
Tags: 3D, ArcScene, multipatch, TIN, topography
Viewing Flood Zones in ArcGIS Explorer, Part 2
This is a continuation of my last post about different ways to access flood zone data for the non-ArcGIS Desktop user.
Method 4: DFIRM Shapefiles
Digital Flood Rate Insurance Maps are available to download from FEMA for $10. They’ve offered a few free samples and Fairfax City happens to be one of them.
The data comes in several formats including shapefile. ArcGIS Explorer can read shapefiles. However, it will not let you add them to your map unless they have a defined projection.
The shapefiles in the Fairfax City DFIRM that I downloaded didn’t have their projections defined. I would assume this is the case with all of them. Luckily, they tell you the projection in the metadata. And luckily, projections can be defined with a file you can create using any text editor.
To find the projection, open the _metadata file in the Document folder. If you scroll down about 2/3 of the way you’ll find the Spatial_Reference_Information section. The most important parts are the Grid_Coordinate_System_Name, UTM_Zone_Number, and Horizontal_Datum_Name. The Fairfax City DFIRM is in Universal Transverse Mercator (UTM) Zone 18, NAD 1983 datum.
I used that information to have ArcGIS desktop create a projection definition file in the format used by all ESRI GIS software (including ArcGIS Explorer). It looks like this:
PROJCS["NAD_1983_UTM_Zone_18N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-75.0],PARAMETER["Scale_Factor",0.9996],
PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
All you need to do is copy and paste that text into a text editor, remove any spaces, then save it as a .prj file. The name before the file extension should match the shapefile you are trying to use. The main DFIRM shapefile is S_Fld_Haz_Ar.shp, the flood hazard zone areas. So the projection definition file should be called S_Fld_Haz_Ar.prj
If you don’t want to copy and paste you can download it and put it the same folder with the shapefile.
FEMA uses UTM for all of its DFIRMs, but they do not always use the same datum. If you download a different one from Fairfax City, you will need to check the metadata for the UTM Zone Number and whether the datum is NAD 1983 or NAD 1927. If it’s NAD 1983, you can use the same text from above as a template to create your .prj file. Just change the two red areas to match what the metadata says:
PROJCS["NAD_1983_UTM_Zone_18N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",
SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-75.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
If it’s NAD 1927, use this template and change the red areas
PROJCS["NAD_1927_UTM_Zone_17N",GEOGCS["GCS_North_American_1927",DATUM["D_North_American_1927",
SPHEROID["Clarke_1866",6378206.4,294.9786982]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],
PARAMETER["Central_Meridian",-81.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
(Remove any line breaks that I’ve entered for readability).
Now, Go to the Add Content button and select Shapefiles… Then browse to the ArcShapes folder and add S_Fld_Haz_Ar.shp. The shapefile starts out looking like this…
…which isn’t very helpful, but just wait. If you right click on the layer in the Contents window, you can change the symbol to something with edges. Now you will be able to see the flood zone borders.
And, if you right click on the layer again and this time bring up the Properties window, you will be able to select certain attributes to show as Popup Content.
(click on the image to see full size)
When you open this dialog box, a list of all the available attributes will come up. You will be able to select which ones you want to appear in a little pop-up window whenever you click on a feature. I picked all of them. Then, in the bottom half of the box you can select one attribute that will appear whenever you hover over a feature with your mouse. I picked FLD_ZONE because this is the most important piece of information. Now, if I type the address of City Hall into the Find box and press enter, I have everything I wanted at the beginning.
There’s a “you are here” symbol, and if I mouse-over I see it’s located in Zone X (not Flood Zone). I can mouse over other areas to see where the nearest 0.2 pct annual chance flood hazard zone is. And if I click in the zone, I can get any more information that exists about it.
Knowing how to use shapefiles in ArcGIS Explorer opens up a world of information. You can watch a free basic overview of the software at http://blogs.esri.com/Support/blogs/esritrainingmatters/archive/2009/12/03/explore-arcgis-explorer-in-a-free-training-seminar.aspx
Tags: ArcGIS Explorer, attribute table, coordinate system, DFIRM, flood zone, shapefile