From the Field: Survey Design and Analysis for Landscape Performance Assessment

By Chloé Nagraj, Master of Landscape Architecture candidate and 2019 CSI Research Assistant at the University of Virginia

Glenstone is a contemporary art museum in Potomac, Maryland that seeks to integrate art, architecture, and landscape into a seamless, contemplative experience for its visitors. Defining and quantifying the impact of this experiential component of Glenstone’s landscape, designed by PWP Landscape Architecture, has been a focal point of the University of Virginia team’s participation in LAF’s 2019 Case Study Investigation program.

Visitor experience is typically among the foremost considerations in public landscape projects. In measuring the social performance of built projects, formal survey design and statistical analysis methods should be more widely taught and practiced within the field. Because measuring the social impact of the landscape at Glenstone was central to our Case Study Investigation, designing a user survey became an integral part of our analysis.

Methodology of Survey Design and Analysis:

I attempted to approach this analysis in two parts: first, the design of the survey, and second, the summary of the results. A key lesson learned through this experience is that survey design and analysis should always be considered in tandem with one another. The study strategy needs to be planned ahead of time, with design informed by analysis methodology. This should include specific goals, careful consideration of the types of data to be collected, response rate, and bias of respondents.

After considering these overarching strategies, I identified the specific topics of interest to be addressed in the survey. In survey design, planning the structure and wording of the questionnaire is one of the most important stages. Ideally, each topic of interest should be covered by multiple questions. As questions are written, they should be edited for clarity and bias. The survey should also be considered as a whole in regards to question order and clarity.

The administration and sampling of the survey also need to be carefully reviewed. In this instance, my questions were added at the end of Glenstone’s existing post-visit user survey, where visitors who reserved tickets online are invited to participate via email. I was not able to give input as to the sampling, ordering, length, or format, as the survey was pre-existing. These factors will affect the results.

Lessons Learned:

A significant lesson learned relates to the analysis of the results. Had I designed both the survey and chosen the statistical analysis methods at the very beginning of the process, my analysis could have been much richer. Time to design both of these critical components should be built into any prospective survey plan.

Additionally, I included mainly Likert-type questions in the Glenstone user survey. A Likert-type question is a 5-7 point scale which allows the respondent to express how much they agree or disagree with a statement. I found that Likert-type questions may not have been the best for my analysis. Had I considered what data analysis I wanted to perform prior to designing the survey, I would have found that binary questions (yes/no, true/false), would be easier to analyze, and would have diversified the types of questions asked for this particular project.

I’ve found that with forethought and research it is valuable and achievable to apply statistical rigor in analyzing data relating to the human experience of landscape through my time spent designing this survey. As landscape architects gain experience in designing, administering, and analyzing prospective surveys on landscape performance, we will be able to better anticipate how people respond to questions, what types of questions lend themselves to performance analysis, and what analytic methods can inform survey design. Applying these techniques and approaching them as rigorously as we can in post-construction site analysis can be useful both in the analysis of built projects and to inform future projects.

Sources:
AAPOR (2015). Code of Professional Ethics and Practices.
AAPOR (2015). Best Practices and Survey Practices.

CSI Team: Emma Mendel & Chloé Skye Nagraj, University of Virginia
Project: Glenstone
Landscape Architect: PWP Landscape Architecture
This CSI Participant update was written in July 2019.

Any opinions expressed in this article belong solely to the author. Their inclusion in this article does not reflect endorsement by LAF.