Implications Of Statistical Process Monitoring For Abet 2000 Program Evaluation: An Example Using Freshman Engineering Attitudes

ABET's new criteria, "EC 2000," has brought assessment and evaluation to the forefront of engineering education. Concomitantly, the focus of ABET's program evaluation has shifted from "what are you (the program) doing?" to "how is what you're doing achieving the desired outcomes (what are your students doing)? 1." In short, accreditation will be concerned as much or more with outcomes rather than inputs or processes. More important, accreditors will be looking at how problems are identified and improvements are made in order to affect the program's outcomes. Therefore, a comprehensive evaluation system should not only accurately measure student outcomes, through proper instrument development and administration, but also possess a well-designed feedback mechanism that allows for the tracking of these outcomes over time. By appropriately tracking outcomes, engineering educators should be able to identify areas for improvement, as well as monitor the effectiveness of programmatic interventions. One basis for such a feedback mechanism is provided by process control charts 2. However, when using data that are either categorical or ordinal in nature, such as to closed-form questionnaires or test scores, naively applying traditional X and R charts can yield erroneous conclusions. This paper will discuss a potential role for process control monitoring of student outcomes for program evaluation and improvement. Using example data about freshman engineering student attitudes, we will present two non-parametric control-charting methods that are appropriate for either survey data or data that are known to be non-normal. We will then illustrate how the charts may be used to identify improvement opportunities and track interventions. Introduction - Use of Quality Control Charts in Engineering Education