Work in progress - work sampling of behavioral observations for process-oriented outcomes

Recent accreditation practices are migrating towards direct measures of student achievement of the eleven enunciated engineering outcomes. Though common examinations are capable of measuring certain outcomes, they are not fully capable of assessing many of the more process-oriented outcomes such as teamwork, problem solving, design, etc. Rich, in-depth assessment methods such as behavioral observation are desirable because they enable us to investigate student learning outcomes and thus evaluate the students' ability to function in the higher level learning domains. Our best current method for doing this (100% observation) requires considerable time and resources. Industry has learned that activities can be assessed using statistical methods that "sample" the observable environment. Work sampling and related methods use probability theory to reduce the amount of time necessary to observe events or activities that do not occur in a systematic manner without loss of information. We are bridging this gap between educational assessment and industry practices by extending these methods to the observation of intervals that capture the cognitive, behavioral and affective domains of student learning outcomes. This paper describes the research involved in developing sampling intervals and provides preliminary results for one process-oriented outcome that of teamwork.