GIS 4035 - Module 10: Supervised Classification for Thematic Mapping

This week focused on creating thematic maps via supervised classification. This involves indicating portions of a raster as training areas to feed to an algorithm. The training areas may be based on previously conducted ground survey, on interpretation of the rasters to be classified or by signature files created by experts and/or expert systems. In any case, the spectral information of the training areas is then used as a guide to classifying the rest of a raster image set, relying primarily on a comparison of multiple spectral bands to interpret their euclidean spectral distance. This later phrase refers to plotting out values of a raster pixel location on two or more bands and assessing how various pixels are grouped.

The map below was created by creating polygons based on spectral 'seeds' grown from a single pixel chosen from various locations on a raster set representing Germantown, MD. The seeds were expanded to polygons with a limited spectral variation from the initial pixel. Multiple locations were chosen for most of the classifications shown on the map, which were then merged after the classification was created using a Maximum Likelihood algorithm (using Bayesian methods to assess each pixels most likely classification). The spectral distance inset map shows the locations that were furthest, spectrally, from one of the classification classes, represented by the bright areas. The brightest areas tended to be fallow fields, which were deemed to have been correctly classified despite the indication of the distance map.

Thematic Map of Germantown, MD

GIS 4035 - Module 09: Unsupervised Image Classification

This week we dealt with unsupervised image classification. This is the process of using an algorithm to differentiate raster image pixels (often across and comparing multiple bands of an image set) in order to create classifications that produce (It is hoped) classifications of pixels that reflect actual ground features. The identification of groupings (and adjustment of classifications) are done after the classification is processed. This stands in contrast with supervised classification where an algorithmically based system is trained before classification by receiving input on areas across the image space that are grouped together into classifications.

The map below was produced by running an unsupervised classification on an orthophotograph of UWF's main campus. The initial classification was set to produce 50 classes. Once those were created, they were each identified as belonging to one of five very simple classes: Trees, grass, building/road, shadow or mixed. The mixed class was reserved for classes that were ambiguous as to their proper identification - encompassing a mix of grass and road. Once all the classes were identified, the classes were collapsed to encompass just those five and the map was created.

UWF Campus Thematic Raster Map

GIS 4035: Module 8 - Thermal Imagery

This week's lab involved manipulating multispectral imagery and using bands in the thermal infrared range to do remote sensing work. While the map below does not display imagery in the thermal infrared (Band 6 in imagery provided by the Enhanced Thematic Mapper Plus suite of satellites), it was used as part of the map analysis performed this week. The map identifies a field network of rice paddies near Guayaquil, Ecuador. Their appearance is somewhat anomalous in the Near Infrared. Healthy vegetation tends to be highly reflective and appear bright in Near Infrared bands (bands 4, 5 and 7 in ETM+), but these fields are dark in those ranges, while appearing like normal, green fields in natural color (where the visible red, green and blue bands are displayed as red, green and blue). In the Thermal Infrared, where captured EMR tends to be the result of heat emissivity rather than reflected, these fields were also intensely dark. During the day (when these images were recorded), water tends to be the coolest material recorded. Vegetation is usually cool, too, but is still somewhat warmer than water. The field'd Thermal IR appearance matched nearby water bodies closely and were darker than areas of healthy vegetation. The inference made was that these were rice paddies - flooded fields planted with rice seedlings. In the NIR and TIR, this gave the fields the appearance of being water, but the seedlings managed to show the signs of photosynthesis to the visible bands (absorbing blue and red, reflecting green to some degree). The latter is probably due to the shallowness of the water.

Rice Paddies in Guayaquil, Ecuador (shown in 'natural' false color)

GIS 4035: Module 7 - Image Preprocessing 2: Spectral Enhancement and Band Indices

This week continued the work with Erdas Imagine and satellite imagery, beginning a deeper delve. Imagery from Landsat 5 Thematic Mapper was provided with the six bands that maintain the same spatial resolution included (Blue, Green, Red, Near Infaread and two mid-level Infrared bands). Several exercises provided practice manipulating and interrogating the images, combining various bands so as to make various features stand out. For example, the most common false color combination where the display layers for Red, Green and Blue are re-mapped respectively to Near IR, Red and Green was given a good workout.

The deliverables this week were created as part of a treasure hunt. The features were described spectrally and had to be located somewhere in the study area of the imagery. Feature 1 represented typical water features, which tend to absorb most electromagnetic radiation in the near infrared band. Feature 2 turned out to be snow, ice and glaciers, which appear bright (high radiometric values) in bands 1-4, but quite dark in bands 5 and 6. Feature 3 was an unusual water feature that had radiometric gradations in 1 - 4 and was somewhat brighter than other water features in these bands. Each feature was displayed in a map with a different combination of bands being used along the RGB display bands.

Feature 1: A Lake

Feature 2: Ice & Snow

Feature 3: Another Lake, with radiometric gradations

GIS 4035: Module 6 - "Spatial Enhancement"

This week continued our introduction to Erdas Imagine with some practice using image filters on raster imagery. This included running various high and low pass filters on an image, as well as edge enhancing and other filters. The map below was produced as an attempt to rectify data errors produced on images from Landsat 7, which sometimes produces bands across an image without any data. A fourier transform was used to reduce the banding somewhat, and then a sharpening filter was run on the resulting image in order to improve the image resolution. The results were not great, but the banding was somewhat reduced compared to the original image.

Moderately corrected banding of Landsat 7 image

GIS 4035: Module 5a - Introduction to ERDAS Imagine

This week's assignment was a basic introduction to ERDAS Imagine, a software package designed for doing remote sensing work, specifically with the editing of raster images in mind. While going through the ins and outs of uploading various rasters, a subset of a larger Land Cover raster was exported from Imagine (after adding an area attribute within the software) which was then used to produce the map below in ArcGIS (thus avoiding an avowed bug in Imagine that would have crashed the system when attempting similar output).

Land Cover Map of a Random Portion of northwest Washington state.

GIS4035: Module 4 - Ground Truthing and Accuracy Assessment

This week's assignment built upon last week's. The Land Use Land Cover assessment that was compiled in Module 3 was assessed in this module. Since actual fieldwork is impracticable for an online course, Google Street Map was the next best thing. Thirty random points were generated in the study area, weighted by land classification so that each class used in the classification received at least one point and proportionally more points were assigned to classes with larger coverage areas. Each point was then located in google maps and the accuracy of the classification at that point itself was assessed and recorded. The map below was then compiled, showing the accuracy result for each point.

The accuracy calculated as a ratio of accurate point classifications to the total number of points assessed is 63.3%.

Accuracy Assessment for Pascagoula, MS Land Use / Land Cover Map

GIS4035: Module 3 - Land Use and Land Cover Classification

This week's assignment was to create a land use and land cover classification for an aerial image that was provided. The study area is a portion of Pascagoula, MS including a good portion of wetlands and open water. We were provided with a basic classification scheme based on the one used by the US Geological Survey and had to identify various land types and create polygons to mark off the various classes as we perceived them on the aerial image. Care had to be taken to keep all the classes at the same scale.

Land Use & Land Cover Map for Pascagoula, MS

Special Topics - Network Analysis & Route Generation

Continuing on from the previous week, this portion of the project involved taking the prepared data files - especially the roads feature dataset that had flood areas, drive times and distances added to its attributes - and generating a network dataset from them, i.e. a dataset that can be used to generate routes from one location within the network to another with minimized costs (usually in distance or time) to find optimal routes. In this case, once the dataset was generated, routes were found for the purposes of evacuating a hospital and also routes were found to be used for delivery of relief supplies to designated hurricane shelters across the city. These routes were generated using expected flood areas as a restriction, although routes for emergency personnel were generated that could move through flood zones, but with a high avoidance factor.

In addition to these routes, the network dataset was also used to generate service areas representing which shelters were closest for any particular portion of the road network across the network. These were shown as polygons around each shelter area, where any part of the network within a particular polygon was closest to the shelter location contained within it.

The map below shows all the routes and the service area generated during this week's analysis. Further routes could be generated at need. Also shown are hospitals, fire and police stations.

Basic Tampa Hurricane Evacuation Map

GIS4035: Module 2 - Introduction to Visual Interpretation.

This week's assignment acts as a basic introduction to performing visual interpretation of aerial imagery. Two digitized aerial photographs were provided which were subjected to some basic interpretations. The first image was used to select areas based on tone from very light to very dark and then also based on texture from very smooth to very coarse. The second image was used to identify features in the image based on various attributes, such as the shape and size of an object, and object's shadow, patterns and associations.

Tone and Texture Identification

Feature Identification

Special Topics - Data Preparation for Network Analyst

This is just the first part of a multi-week project. The conceit of the project is that optimal evacuation routes need to be generated for the Tampa, Florida metro area just days ahead of an expected hurricane. These routes will then be produced in maps for use by both the general public and by emergency personnel. This week's work was simply to create the base map from which a network dataset will be generated that will be used to create the routes in upcoming weeks. This involved prepping feature datasets for that upcoming work and also generating a basic map that shows the location of emergency services and also the expected flood zone of the area. This prep also included creating attributes in feature datasets that will be used for generating a network dataset.

Base Map of Tampa Metro Area with Expected Flood Zone & Emergency Services

Catchment Area Analysis in the Valley of Oaxaca

The final project for applications in archaeology involved a research project, the default being a catchment area analysis for a portion of the Oaxaca Valley. Estimates were made of collection unit populations for various periods in a given set of (digitized) grids from the Valley of Oaxaca Settlement Pattern Project performed by Blanton et al. in the late 1970s. From these, an estimated target yield for maize production needed to satisfy food security for these populations was arrived at and compared to an estimate of the maximum maize yield from a catchment area of 1 kilometer around each collection unit as generated by a buffer in ArcGIS.

Catchment Areas for Guadalupe Period

Catchment Areas for Early Monte Albán I

Catchment Areas for Late Monte Alán I

Catchment Areas for Monte Albán II

GIS Programming Module 10 - Creating Tools

This week's work was to create a toolbox and script tool in ArcGIS so as to leverage an existing script via the ArcGIS UI. The script was modified such that parameters were sent to it via a dialogue window in ArcGIS and to send any messages that had been directed to StdOut to the ArcGIS results window instead. The end result is a script tool that can be used easily within ArcGIS by users without any familiarity with scripting for ArcGIS and which is also easy to share with users on other systems.

Input dialogue window generated by the script tool

Results window showing output from the script

GIS Programming Module 9 - Debugging

This week involved debugging code. Error ridden code was provided. Syntax errors and logic errors were discovered and cleaned up. Code then ran. Exception statements were also used to allow code to complete an execution without breaking out in the middle.

Script executes successfully

Script executes successfully

Script encounters an error, prints an error, then continues to execute successfully

GIS Programming Module 8 - Rasters

This week involved manipulating raster files, including performing map algebra via python. The output below took two separate rasters - one displaying land cover and the other elevation - of the same region and transformed the data therein to create a new raster. From the land cover raster, three types of forest were identified and reclassified into a single class and extracted. From the elevation raster, slope and aspect manipulated such that a desired range for both were identified as '1' while all other pixels were assigned '0'. The final raster output, below, is the result of a union of all the data transformations using boolean algebra to generate it.

Forest Cover with desired slope & aspect in yellow

Land Cover Classification Analysis

This week's maps were produced by running spatial analysis tools to run, first an unsupervised raster classification on an ortho-rectified aerial photo and then a supervised raster classification (wherein a point file was created that identified various points in the raster as having a particular classification. The goal of this would be to create high contrast classes of land cover in aid of using remote sensing as an archaeological prospection tool.

The results indicate that while this type of analysis may be helpful in the right circumstance and with the right kind of data, it is not necessarily the most useful approach. Land classes need to be differentiable enough to prevent mixed classes that cross different classes, causing the resulting classification scheme to obscure more than reveal.

Unsupervised Raster Classification

Supervised Raster Classification

3D Modeling for Archaeology

This week's assignment involved composing interpolated subsurface soil layers (interpreted as the interfaces between topsoil and cultural layers, then cultural and natural layers) from shovel test data. From a point shapefile of shovel tests with the depths to cutural and natural recorded as attributes, depth lines were composed for each test location in ArcScene. The data was also used to create Inverse Distance Weighted surfaces in ArcGIS that were then transferred into ArcScene and extruded to the correct depths.

Also, a flythrough video was generated through the depth lines of the shovel test points.

shovel tests

interpolated layers

depths of proposed fence posts calculated from IDW surfaces.

GIS Programming Module 7 - Working with Geometries.

This week's script used a cursor object to run through all the rows of a shapefile, collecting their OID, their geometry and a NAME field. These were used to print out the data from the shapefile into a clear text file. The resulting file displayed a row for each point of each feature in the shapefile in a space delimited format. The fields in the text file were:

• Object ID number
• A vertice number, as counted up from zero for each feature
• The vertice/point's X coordinate
• The vertice/point's Y coordinate
• The NAME field

Screenshot of resulting text file

Using Surface Interpolation to interpret Archaeological Survey Data

This week's assignment went over the details and intricacies of importing various types of survey data into ArcGIS and performing spatial analyses upon them, specifically surface interpolation of artifact density patterns. Various types of interpolation were performed on two sets of data, one on the scale of a occupation site where shovel tests were performed across the site and a second on a regional scale looking at the density of collection areas.

The work focused on the nuts and bolts of performing the analyses and did not touch on, nor explain much of, how best to chose analysis methods and the parameters thereof to provide an accurate and useful analysis.

Various interpolation method examples shown across two regions at variable scales.

Manipulating Spatial Data (GIS Programming Module 6)

This assignment involved using the arcpy module in Python to access spatial datasets in order to find and manipulate the data within. The script whose output is shown below creates a new file geodatabase, populates it with shapefiles from an extant data directory and then creates a dictionary that holds population values using city names as keys, specifically cities that possess the 'County Seat' attribute from the FEATURE column in the attribute table for the 'cities' feature class.

script output 01

script output 2

Digitizing Archaeological Datasets

This project spanned two weeks and was rather intensive. Scans of survey grids from the Oaxaca Valley Settlement Survey Project were georeferenced and the spatial data from the surveys were digitized into attribute tables of shapefiles. The task of georeferencing was made difficult by the poor quality of the scan of the survey grid and also by difficulty in finding a suitable basemap of the Oaxaca Valley onto which to rectify the grid. Eventually a solution was found, though, and the grid was tied in. Afterwards, the separate grids for land cover and collection surveys were tied into the grid and then their data was digitized. Finally, each grid was laid out with land cover and survey results and a map was created, or three in the case of Grid N2E3, which had 3 grids of survey collection data due to the density of occupation.

Grid Locations

N2E3a

N2E3b

N2E3c

N2E4

Geoprocessing in Python

This week's script called up functions from the Python arcpy module for geoprocessing tools from ArcGIS. The scripting was straightforward. A point vector shapefile had had processed to add XY coordinates to features, then each feature was given a 1000 meter buffer, then the buffered features were dissolved to form a single feature.

From the completed process summary:

1. Added the import call and environment settings (workspace & overwrite = true)

2. Confirmed parameters and syntax for the three geoprocessing functions in the ArcMap python window

3. Created variables for shapefile inputs and outputs

4. Wrote up the calls for the three geoprocessing functions one at a time, printing the results via the GetMessages() function each time. Ran code after each function call was written.

5. Added comments as I went.

Geoprocessing messages were directed to stdout:

Python Fundamentals II

This week's assignment involved debugging some code provided and then extending it with some directives designed to provide practice with using conditionals and loops. The first part of the output below (names listed as winning and losing) was provided with some bugs in the code to prevent it from running. These were found and fixed. The second part of the assignment involved generating a list of 20 random integers that ranged from 0 to 10, printing the result and then pruning a single integer from the generated list. The list was generated from a simple while loop (as per instructions) that appended a new random number to the list for each iteration. The break state for the loop was when a variable incremented each time reached the value of 20. Python's list functionality made manipulating the list quite easy and only took two lines of code to remove all instances of a particular integer.

Code output

Georeferencing 1785 Map of Macao

This week the assignment involved georeferencing (as best as possible) an historic map. While the map couldn't be georeferenced with utter precision, using a spline algorithm, it could be placed well enough to provide a recognizable regression from the current state of the area.

Macao, 1785

Python Fundamentals I

This week's assignment involved writing a simple script that assigned my full name to a string and then manipulated it in various ways. The output from the script was to be my last name and the number of letters in my last name, tripled. This was a basic introduction to various data types, data structures, functions and methods.

The screenshot below shows the script output from running the script four times, including two times with a different name assigned as the initial string.

Script Output

Dress Rehersal for Presenting Historic Maps

This week's assignment was simple. We were given a raster of an historic, late 19th Century topographical map of coastal Massachusetts that was already mosaicked and geo-referenced. From this, a basic map was created that showed the Boston area, located Paul Revere's home and presented Revere's portrait and a census record from his household recorded in 1790.

Paul Revere's Home

Setting up a basic ArcGIS model and converting it into a script.

This assignment was fairly basic. A simple geoprocessing operation was constructed using the ArcGIS Model Builder, which was then exported to a standalone python script.

The process took shapefile with soil data and clipped it to encompass just the area of a small basin defined in a separate shapefile. The clipped layer was then run through the selection tool to identify poor soils via an entry in the attribute table and then the selected features of the layer were erased using the erase tool. The result is the layer depicted below.

Only the best soils.

The map produced this week was based on data pulled form MEGA Jordan, an online spatial database for Jordanian archaeological sites. This resource was designed and implemented in order to aid in the management of archaeological and heritage resources in Jordan, is based on free and open source software and, being well supported by the government of Jordan is comprehensive and responsive. All in all it seems to be an excellent resource for heritage management.

This week's course content also touched on the ethics involved in maintaining archaeological databases (as well as other types of archaeological resource output). As recorded data is often all that is left of an excavated archaeological site, data protection is critical to preserving the archaeological record. Also, site protection and preservation requires that archaeological databases, spatial or otherwise, be kept secure, especially for vulnerable sites.

Jordanian Archaeological Sites

Brief Introduction to Python Interfaces

This week's assignment for GIS Programming was simply to access various Windows based ways to code in Python (which ESRI has adopted and integrated deeply into their ArcGIS environment). This included the default installation of IDLE for Python, PythonWin and the Python IDE environment that's a part of ArcGIS 10. After issuing print commands in various ways - via the IDE CLI, via *.py script, etc. - a script was run out of PythonWin that set up a standardized directory hierarchy for future modules, the results of which are depicted below. The only other deliverable was a barebones process summary.

Screenshot.

The first assignment for Applications in Archaeology involved creating a map of Chicago that showed the extent of the 1871 fire as well as landmarks that pre- and post-dated the fire. Much of the assignment focused on performing attribute queries as well as clipping layers.

The Impact of the Great Chicago Fire of 1871

Analysis of Proposed Electric Power Transmission Line Corridor

This project involved analyzing the suitability of a proposed power line transmission construction corridor across several metrics. The corridor's interaction with conservation lands, with residences and with schools were all looked at with an eye toward minimizing its impact upon them while also limiting the construction costs for the transmission line as much as feasible while still respecting the other constraints.

• Bobwhite Manatee Transmission Line Corridor Selection & Proofing Project [Powerpoint File]

• Slide Breakdown to accompany powerpoint presentation

Mean SAT Scores & Participation for 2013 by State

This project involved displaying two datasets on one map, the mean SAT scores by state for 2013 as well as the rates of participation. The scores are displayed as a choropleth map with the participation rate overlaid as bar graphs that show the ratio of test-takers to non test-takers for each state. Care was taken that the bar graphs for each state were clearly assigned and readable.

There seems to be a clear inverse correlation between rates of participation and mean scores for each state. So much so that for the highest two classes for SAT scores (i.e. 1601 and above), the highest participation rate reaches only 13% (Ohio). Meanwhile, the states that have 100% participation or close to it (Idaho, Maine, Delaware) have mean scores among the lowest in the country. All three are in the bottom five along with another high participation region - Washington DC at 81% - and a middle of the road state when it comes to participation, Hawaii.

The low participation states seem to have a population of motivated test takers who have a vested interest in the outcome of the test, while the high participation states have made taking the SAT mandatory for all graduating high school students, thus diluting their results with a large volume of students with no stake in their test results.

The primary takeaway from this map, then, is that mean SAT scores by state is not a useful metric for comparing educational outcomes across states.

Mean SAT score totals by state for 2013

Georeferencing & ArcScene

This week's primary assignment involved taking aerial photos that had absolutely no geographical reference metadata, importing them into ArcGIS and georeferencing them with existing map data. This was familiar territory for me, as this has been the main activity I've done with GIS software as an archaeologist. I was provided with two rasters that showed portions of the UWF campus as well as projected vector files for campus buildings and roads. By associating points on the rasters with points in the vector layers, I rectified the raster layers in short order, getting them aligned with their vector counterparts. The rasters were in fairly good order, so there was not a lot of error in their placement, even though one of the rasters had been deliberately warped to make georeferencing it somewhat more complicated.

A secondary task this week was to produce an exaggerated 3D style map based on the georeferenced map I created using ArcScene.

Georeferenced UWF Campus Map

UWF 3D Map