Selection into Shipbuilding Occupations when Dealing with Missing Data: 411 Board #232 May 31 9

Building battle ships involves transforming sheets of steel into many shapes and joining the segments using welding, brazing, torqueing, and other methods. PURPOSE: To design and validate selection tests for 30 jobs (e.g., shipfitter, joiner). One of the challenges was prevalence of missing data due to intermittent availability of the workers in the shipyard. Thus, this research used alternative methods to identify statistically the tests predictive of job performance for 30 occupations. METHOD: A job analysis survey completed by 629 workers identified the essential tasks for each job. Research staff identified the physical abilities required to perform essential job tasks. Researchers conducted a criterion-related validity study that included ten predictor tests and five criterion measures. The criterion measures included performance of tasks onboard a ship, along with supervisor evaluations of physical job performance. The sample included 197 men and 47 women across 24 of the 30 jobs. RESULTS: Validation data yielded a model (R2=.59) that consisted of lift/carry climb stairs, arm endurance, container lift, and plank. Further analysis showed the test battery was fair to protected groups (e.g., sex, age). Due to varying shift schedules, missing data occurred for the predictor tests and criterion measures for many subjects. Listwise deletion in the regression analysis resulted in a final sample of 155. Although statistical power was high for this sample, we conducted a Full Information Maximum Likelihood (FIML) analysis to determine whether the missing data affected the conclusions. FIML used a maximum likelihood approach to estimate the missing data based on all available information for a subject in an unbiased manner, rather than not replace or impute missing data. CONCLUSIONS: Comparison of squared multiple Rs for all ten tests for the FIML (0.61) and original (0.59) analyses found a small difference with FIML accounting for 1.3% more variance. The FIML standardized beta coefficients with the highest values were the same as the original regression analysis, thus confirming the original results. We established separate passing scores by job and test using information from the validation and job analysis results. Each job’s test battery contained only tests and passing scores relevant to the job.