In Washington, D.C., 3.5 million commuters get around the city on Washington Metropolitan Area Transit Authority (WMATA) trains and buses. In 2008, there were 215 million rail trips and 133 million bus trips in its 1,500-square-mile area.

WMATA is liable to the National Transportation Safety Board and other federal mandates to make sure that its bus drivers and train operators are properly trained in safety and other procedures. For years, WMATA has tested its drivers, operators, mechanics and station managers with paper-based tests. Many drivers and operators had to change their schedules (and WMATA had to find replacements) so that they could go to training and take their tests. A lot of time got wasted while instructors manually graded those tests and manually entered their scores into a system to track their training.

Out of WMATA’s 10,000 employees, roughly 8,000 are operations employees: drivers, operators, track walkers (employees who fix the tracks), station managers, and mechanics. All these employees have to learn WMATA’s procedures and take federally mandated safety training. For example, track walkers have to know hand signals for staying safe on the tracks, while station managers have to know how to deal with emergency situations, blood-borne pathogens, and other safety-related situations.

WMATA began using Perception to administer ungraded quizzes and pass/fail tests in the initial training for drivers and operators, and in the recertification training for mechanics, track walkers, and station managers. For example, train operators must take 5 quizzes and 1 pass/fail test during their first two weeks of training.

WMATA maintains separate item banks for each employee category, and pulls items randomly to create new tests for each class. Employees can take their tests from any location, but all graded tests are proctored. To access their tests, employees log in from any WMATA computer to the learning management system (LMS), TrainingServer. The test is immediately graded and the score is stored in TrainingServer. "The scores are also stored in Perception, which provides redundancy and enables us to view each employee’s response on a question-by-question basis," says John Frensilli, a computer-based training specialist for WMATA. "The instructor doesn’t have to grade the tests, they can use class time to review the test, and there’s no longer a two-week delay for transcription of scores.”

Instructors from WMATA’s training department work with curriculum developers to create questions that are both technically accurate and pedagogically sound. They use the Item Analysis tool to make sure all the questions are performing properly and to make sure the curriculum matches up with the test questions.

Frensilli says WMATA also uses the reporting tool when the NTSB audits training records for compliance with federal safety training requirements. “It’s easy to run reports that show that x number of people completed training this year, or determine who’s in compliance and who isn’t,” he says.

Train operators are tested the most in Perception, followed by bus drivers. Station managers and mechanics are only taking recertification tests in Perception, and take other tests on paper. But eventually, Frensilli says, Perception will replace more of those paper-based tests.

“We’re pleased that there’s no need to pull people or change their work schedules,” Frensilli says. “We’ve reduced our costs, keep training periods shorter, train a lot more people for recertification, and easily report on our compliance.”