Selecting Assessment Tools

Interpretation

Interviews & Focus Groups

Assessment Series 2012

CampusLabs

Interpretation

Interpretation or analysis of data could be the most exciting and rewarding dimension(s) of program assessment.  However, it is also the riskiest portion of the assessment. All program evaluation/assessment is vulnerable to misinterpretation. No matter how “quantitative” the data and calculation in your assessment tool, it is under the mercy of the quality of qualitative interpretation.   No matter how “accurate” the research data “calculation” is, data could be rendered useless with poor qualitative interpretation.

Surveys

Most student affairs units will use some types of surveys. In particular, they will use a version of a student satisfaction survey. We believe that surveys are excellent ways to gather information that describes, compares, or explains knowledge, attitudes, and behaviors.  We have found that THE SURVEY KIT by series editor Arlene Fink is an excellent resource for conducting surveys.  We highly recommend that you acquire and study this resource, and therefore we do not attempt to supply information in much detail regarding surveys.  It provides some of the more sophisticated information about using surveys that can guide your survey development.

THE SURVEY KIT is available through SAGE publications at: SAGE Publications, Inc., 2455 Teller Road, Thousand Oaks, CA 91320 (email: order@sagepub.com)  

Types of demographics to gather

• Gender
• Ethnicity/race
• Age
• Year in College
• Classification
• First-generation college attendee
• In-state or out-of-state
• Campus Housing

Types of survey questions

• Closed-ended
• Open-ended
• Multiple choice
• Check all that apply
• Ranking
• Closed-ended followed by open-ended

Examples of methods units may use to gather survey information

• Immediately after the student receives service or participates in an event, such as a workshop. Every time a student uses the service s/he is asked to complete a survey.
• Conduct surveys for a designated period of time (e.g., three months).  Every student who uses the service will be asked to complete a survey at the time the service is given.
• Give surveys at the end of some designated period such as at the end of the semester, at the finale of a program, or near the end of the school year. 

Tips on conducting surveys

• Before using a survey, pilot it on a small sample of a similar audience.  Then modify unclear or poorly worded questions or delete those questions that do not seem to elicit worthwhile information.
• Keep the survey as short as possible. Be selective about the questions you ask. 
• Survey should be easy to read. (on lightly colored paper, legible font, photocopy quality).
• Survey should include information at the beginning and the end about when and where survey should be returned.
• Decide whether the survey will be anonymous (without the student’s name or ID number) or ask the student for their name. For student affairs units, most of the time surveys should not ask students for their names.
• Use a combination of open and close-ended types of questions.
• Each question should ask only one question.
• Organize the questions into a meaningful flow – usually starting with background information on the student (demographics), and ending with questions about the future. More sensitive questions should be asked in the middle of the survey, not at the beginning.
• Consider offering incentives to encourage students to return the survey (placing their name in a drawing for a gift certificate, book, privilege, etc.). If you do this, if you want the surveys to be anonymous, you will need to keep the student’s name separate from the specific survey. 
• When implementing a student satisfaction survey, your office may want to prepare a Frequently Asked Questions (FAQ) sheet to inform the students about the survey.  Some sample questions you may want to answer on a FAQ sheet are listed below:                      
• What is this survey about?                           
• Who is this survey from?                           
• Why should I take the survey?                           
• When can I fill out the survey?                           
• How do I fill out the survey?                           
• Who will read the answers to this survey?                           
• How is my privacy being insured?                           
• What is the point of this survey?                           
• Who do I contact for more information?

• Remember, the main purpose of undertaking a student survey is to obtain some information from your students on their experience of the unit. If implemented with much thought and effort, a survey can help you and your staff immensely. The goal is to have all cohorts complete the survey. If 70 percent or more of your students complete a questionnaire then you can be confident that the information gathered will be reasonably representative of the class as a whole. If the remaining students completed the questionnaire then it is unlikely that the results would change considerably.  If only 50 per cent complete the questionnaire then your confidence in the results being representative must be less.  If less than 50 percent complete the questionnaire then the information collected is not particularly useful for making decisions with regard to changing your unit.   For units where the response rate is less than 50 percent of enrolled students, results will be qualified with the statement that - these results cannot be taken as representative of this class of students and can only be said to reflect the views of those students who completed the questionnaire.