Posts Tagged ‘geodatabase’
Creating an XY Domain that covers your data extent
Problem: When loading data into into a feature class, you get the error “the coordinates or measures are out of bounds”
This issue most often arises for me when I am in ArcCatalog using the simple data loader tool. It can also arise when trying to import a feature class into a feature dataset, when creating new features during an edit session in ArcMap, or when performing spatial adjustment to move features to a new location. In each case, the error occurs when the extent of the feature class or feature database is not large enough to contain the new or moved features. The solution would appear to be “increase the extent,” however, it is not possible to change the extent once a feature class or feature dataset has been created. You need instead to create a new one with the correct extent. I will show you how to do that, returning to the example of the Oceanic layers I used in my post about preparing layers for merging with model builder.
Today, I would like to merge all my city point shapefiles into one feature class in my master Oceanic geodatabase. I have city points for Australia and Papua New Guinea. If I merely import the Australia shapefile as a feature class, and then attempt to load the Papua New Guinea points into that feature class, I will get this error message:
One of the points is out of bounds! By displaying the layers in ArcMap, it’s easy to see what happened. When I imported the Australia shapefile, the spatial domain was defined as the extent of that layer. The one Papua New Guinea point that didn’t get loaded fell too far east of that domain.
A good rule of thumb is: if you have a dataset that covers your entire domain, import that first and then load the other datasets into it. However, you often won’t. In that case you are going to have to take one or two extra steps.
Single step method (good enough for most purposes):
Create a new feature class. ArcCatalog will ask you to chose the projection of the feature class, and when you do, the extent will be set to the all of the mathematically possible values in that coordinate system. In other worlds, the whole world (and then some). ArcCatalog will then allow you to chose a schema for the new feature class. Import the schema from one of the layers you are going to load in.
You will now be able to load all of your layers into the new feature class without any out of bounds errors.
Two step method (better):
Add all of your layers to ArcMap. Then draw a box around all of the features and save it as a separate polygon shapefile.
This polygon shapefile will be used to precisely set your x,y domain. In order for this to work, the projection of the layer must be set to unknown. Clear the projection if it has one defined. Now, create a new feature class in ArcCatalog. When it asks you for the projection of your new feature class, import the projection of your polygon shapefile.
If this polygon has a defined projection, ArcCatalog will just do what it did before and set the extent as all mathematically possible values in that coordinate system. But if you import an undefined projection, ArcCatalog sets the domain as the spatial extent of that dataset. That’s exactly what you want. You’ll see an additional screen that shows you this is happening and even allows you to change the x,y domain values if you want.
Here, you can see what the domain looks like if it is defined automatically by the GCS WGS84 coordinate system.
And here is what is looks like when defined by my domain polygon with an unknown coordinate system.
Why is the second way better? It is preferable to have as small a domain as possible because a smaller domain allows for a greater resolution. Resolution is the minimum distance between coordinates. A higher resolution means that locations can be placed more precisely, because their coordinate positions are allowed to have more decimal places. Because data storage is finite, there will always be a trade off between extent and resolution. In most cases, the resolution gain will not be noticeable, especially if you are using a post-9.2 high precision geodatabase. But I put it here as a best practice.
Once you have loaded all your layers into your new feature class (error free!), define its coordinate system as your last step. This feature class is totally optimized for the data it’s holding.
Tags: coordinate system, data loader, domain, geodatabase
Removing Empty Feature Classes with Python
The Problem: You have a geodatabase with a lot of empty feature classes
This often happens if the geodatabase has been created from a template of standard themes, but this particular version doesn’t contain information to stick into all the available slots. It can be difficult to get a handle on what’s actually there with all the empty feature classes clogging up space. It can take up extra processing time or cause errors depending on what you’re doing with the data later. And, it’s just bad organization. The best practice is to get rid of them. You can do so by moving down the feature class list in ArcCatalog, checking for empties and deleting them one by one. But with large databases (the kind that need trimming down the most) this can take a lot of time.
The slow manual method:
The fast automated method: use this python code that an associate shared with me!
Copy and paste the following into any text editor and save it as a python .py file.
##############################################################
## Delete Empty Feature Classes from Multiple Geodatabases
##
## This script loops through all geodatabases within a directory
## and removes empty featureclasses from each geodatabase.
##
## User must hardcode directory. Run this script in IDLE.
##
## Python Version 2.4
##############################################################
# Create the Geoprocessor object
import arcgisscripting, os, sys, string
gp = arcgisscripting.create()
# Load required toolboxes...
gp.AddToolbox("C:\\Program Files\\ArcGIS\\ArcToolbox\\Toolboxes\\Conversion Tools.tbx")
# Set workspace
gp.workspace = "C:\\Documents and Settings\\user\\Desktop\\gis\\"
# list personal geodatabases in the current workspace
listmdb = gp.listworkspaces("*", "access")
mdb = listmdb.next()
# loop through the personal geodatabases
while mdb:
print mdb
gp.workspace = mdb
# list feature datasets in personal geodatabase
listfdset=gp.listdatasets()
fdset=listfdset.next()
if not fdset == None:
gp.workspace = mdb + "\\" + fdset
# list feature classes in the personal geodatabase
fcs = gp.listfeatureclasses()
fcs.reset()
fc = fcs.next()
# loop through the featureclasses
while fc:
# print fc
# if the feature class table has no records, delete the featureclass
if gp.GetCount_management(fc) == 0:
gp.Delete_management(fc)
print fc + "deleted from" + mdb
fc = fcs.next()
mdb = listmdb.next()
The only thing you will need to change is the directory under # Set workspace. Put the path to your geodatabase(s) in between the gp.workspace = “” quotes, remembering to use two slashes \\ in between folder names.
This works with personal geodatabases. If you are using file geodatabases, go to the next line under # list personal geodatabases… and replace “access” with “FileGBD” between the gp.listworkspaces(“*”, “”) quotes.
Now, start Python IDLE, which ships with the ArcGIS suite. Open up your .py file. Then, select Run –> Run Module (or hit F5). This is what you will see:
“
“
It may take some time to run, but that is time you can spend doing other things. When it’s done, those empty feature classes are history.
You can edit, save and rerun the .py file as many times as you need if your geodatabases are spread between different directories. (Or, for even more automation, copy them all into the same directory before you start, and let the script loop through them).
Python is an extremely handy tool. If you are interested in learning more about it, see the resources below.
- HowTo: Get started writing Python geoprocessing scripts in ArcGIS 9x (ESRI Support)
- A Guide to the Python Universe for ESRI Users (PDF File)
Tags: attribute table, automation, geodatabase, python
Adding file size and date modified to ArcCatalog
The problem: You need to discover the file size of an individual feature class within a geodatabase.
In the world of shapefiles, it was relatively easy to do this. You could open up Windows Explorer and add up the sizes of the component shapefile parts:
.dbf = attribute table
.shp = feature geometry
.shx, .sbn, .sbx = various spatial indicies
.prj = projection (optional)
.xml = metadata (optional)
When we migrated into personal geodatabases, you could at least see the size of the entire database by looking at the Microsoft Access .mdb file. You couldn’t drill down into individual feature classes, however. Then, when the file geodatabase came along, things got even worse. The characteristic feature of this data type is storage as file folders. And yes, the file geodatabase looks like a folder in Windows Explorer. But the contents of that folder are unintelligible.
The image below shows how these three most commonly used data types appear in Windows Explorer:
A peek inside the FileGeodatabase folder:
ArcCatalog, the file navigation system specifically for GIS data, is clearly the superior option here. However, just like with Windows Explorer (where you have to switch to detail view to see more than a list of thumbnails) it needs some tweaking before you can get the information you want out of it.
Here’s how: Go to the Tools menu and select Options, then the Contents tab. This will give you a list of the details that show up on the Contents screen when you are navigating through your file system. Name and Type are the only ones checked on by default. You will at least want to add Size and Modified.
Note: Date modified is useful to display for so many reasons, not the least of which are
- Gives a ballpark idea how current the dataset is, if you don’t have that information elsewhere
- Helps you differentiate between different versions that may be named the same thing
- Sorting by date modified is a one click way to gather together everything you were last working on
While you’re there, take a look at what else is available. The settings you select here will remain every time you open up ArcCatalog. Ever since I stumbled upon this, it is one of the first things I do when moving to a new machine.
I’ve taken ESRI’s USA major roads data and saved it into a shapefile, personal geodatabase, and file geodatabase. With these new settings, here’s how they appear in ArcCatalog’s Contents screen.
This is a good place to point out the advantages of using file geodatabases. As you can see, they use dramatically less space to store the same amount of data. This means faster load times, as well. The other two won’t be going away any time soon: Shapefiles are widely used and good in a pinch, and personal geodatabases are nice if you like using Access to edit attribute tables. But for long term file storage, file geodatabases really are the way to go. As a bonus, you can compress them as read-only, which saves even more space.
Tags: geodatabase, settings