On the job
Featured eBooks
Connecting state and local government leaders
A customized tool lets course instructors and supervisors at the Army Research Institute for the Behavioral and Social Sciences design surveys that fit their own needs.
A customized tool lets course instructors and supervisors at the Army Research Institute for the Behavioral and Social Sciences design surveys that fit their own needs.That eliminates the cost and hassle of manual survey tabulation by a shrinking staff. And it lets Army leaders compare results servicewide.The institute's Occupational Analysis Office supervises course evaluations for the 29 Army Training and Doctrine Command schools, said Ron Stump, the office's head.TRADOC schools teach specific work skills to enlisted personnel and officers. The Occupational Analysis Office is responsible for determining whether the courses are teaching the right material and teaching it effectively.TRADOC instructors use Autogen, developed for the Army institute by Raosoft Inc. of Seattle, to develop their own surveys, post them on the Internet, and collect and analyze the responses.As program manager for Autogen, Stump works with Raosoft when command users want enhancements to the application. Stump's staff also feeds data collected from TRADOC to other Army officials in charge of occupational and training oversight.The first version of the tool made its debut in April 2002, said Elizabeth Brady, the institute's senior research psychologist.Autogen handles two kinds of surveys. The job analysis survey examines the duties that soldiers perform in their current jobs to reveal long-term strengths and weaknesses of the training they received, she said.The training evaluation survey is tied to specific TRADOC courses. Some courses teach specific skills, and others teach such subjects as leadership skills, which noncommissioned officers need for promotion.TRADOC schools offer hundreds of courses, but Stump's office now has only five staff members. They 'could not ever keep up with the number of surveys that would be required' without Autogen, he said.Back when the Occupational Analysis Office had 40 to 50 staff members, it developed all the surveys for the TRADOC schools, collected the results on paper, analyzed them and reported the findings back to the schools.'Through the years our program continued to get cut, and we weren't able to provide support,' Stump said.Now the schools are responsible for developing their own computer-assisted surveys with the institute's consultation.'We're more into the consultant phase as well as training to get people up to speed on the software,' Stump said.After downloading Autogen and clicking to start the 'Create a survey' tool, a TRADOC official is led through a series of browser screens, Brady said.In many cases, each screen informs the author what information must be gathered to design the next page of the survey. Most questions are framed in drop-down boxes.'The key is that it's standardized, so we don't have 29 schools going in 29 different directions,' Stump said.Each survey must have specific background variables to validate data and permit sorting of the results by rank and reserve or active status.Next the program requires setting up a task-based inventory for a particular job or course. Survey authors don't have to devise their own rating scales because Autogen has its own tested scales.Finally, authors can choose from 25 questions to meet special requirements for their courses or employment tracks.'You don't have to be an expert in statistics or survey methodology to put the survey instrument together,' Stump said. Army leaders can compare data from multiple schools or compare tasks from two occupations to see if job categories can be merged.Before Autogen, one school would ask about tasks in its course survey, and another school with a different agenda would develop a survey that didn't cover tasks. So Army leaders could never directly compare the surveys.'One of the beauties of Autogen is that it's standardized, and there's not a lot of room for people to go off and do their own thing,' Stump said. Before Autogen, he said, his office used to mail out 20-page survey booklets to the schools, and later diskettes. But many respondents are mechanics who work with gasoline, oil or other messy substances. 'Diesel fuel and scan-form sheets don't go well together,' he said.
Army schools automate surveys
One direction
Elizabeth Brady, senior research psychologist at the Army Research Institute, and Ron Stump, chief of the institute's Occupational Analysis Office, say automating surveys helped them cope with staff cutbacks.
Olivier douliery
One direction