From the Field: Bumps in the Performance Evaluation Road
By Erika Roeber, Bachelor of Science in Agriculture Candidate, South Dakota State University
Driving across South Dakota’s never-ending interstate, you are likely to hit a few bumps. Evaluating the performance of the three projects we are studying as part of the Landscape Architecture Foundation’s Case Study Investigation (CSI) program was no different: smooth sailing at first with the planning of our economic and environmental benefits. The social benefits brought our first bump when our research team decided to conduct user surveys for each of the sites.
All survey research involving human subjects requires approval from a university’s Institution Review Board. To get IRB approval at South Dakota State, each researcher needs to have a Collaborative Institutional Training Initiative (CITI) certificate. The faculty Research Fellow was already certified, but I had to complete a course with multiple readings and quizzes — in all it took about ten hours to complete. After that, the process of submitting our proposal and getting approval for our surveys only took 1.5 days. Our expert tip: use exempt status and only have adults as survey participants; this makes IRB approval a smoother process.
Being native to South Dakota, I know there is more than just one bump along the highway. The Mount Rushmore Visitor Center Redevelopment presented a huge obstacle when it came to doing user surveys. In order to do research on any federal land, including all national parks and memorials, researchers must first obtain a National Park Service (NPS) research permit. This process takes anywhere from one to six months. Once approved, the researcher can use scientific equipment to gather data (i.e. thermometers, water samples, etc.), and they can distribute up to nine surveys on-site.
If the researcher wants to disperse more than nine surveys, they must complete an additional application through the Paperwork Reduction Act (PRA) and PRA Clearance, a term used for the process of obtaining approval from the Office of Management and Budget (OMB) for federally sponsored data collections as required by the PRA. This rule was put into effect due to increasing complaints from the public about duplicate and lengthy federal government data collections. The PRA clearance process can take anywhere from six to nine months to be approved. Upon learning this, it was evident that on-site user surveys would not be feasible within the CSI timeframe.
Instead, we decided to post our surveys online through social media. A member of the Mount Rushmore staff indicated that we would not have any regulatory difficulties doing our surveys this way. However, upon further research and after the survey had been posted, I discovered that online surveys do require PRA clearance if specific questions are asked. So we should have gone through the PRA process after all. This approval is not necessary for very general invitations for public comments and suggestions.
Through all the bumps in the road, we were still able to gather valuable data and information about the Mount Rushmore Visitor Center Redevelopment. Alternative approaches had to be taken, but they were doable given the variety of information that had been collected from past research. Despite the challenges, federal land represents an exciting opportunity for landscape performance research.
Research Assistant Erika Roeber and Research Fellow Matthew James, Ed.D are participating in LAF’s 2015 Case Study Investigation (CSI) program and working to evaluate the environmental, economic and social performance of three public projects in South Dakota. Any opinions expressed in this article belong solely to the author. Their inclusion in this article does not reflect endorsement by LAF.
