A Pilot Study to Develop VR Case Review Instrument for WIOA Performance Measure Data Collection

In 2015 the Rehabilitation Services Administration implemented the Workforce Innovation and Opportunity Act (WIOA). This legislation changed the way Vocational Rehabilitation (VR) services were provided in all state agencies, and also introduced new performance measures the RSA would use to verify compliance. The act shifted VR focus from quantity to quality, requiring a major change in how data was collected, interpreted, and reported.

This project focused on developing an instrument that would incorporate performance measures into case documentation; provide a consistent method for interpreting quality; and that would gauge compliance with WIOA, state, and internal agency regulations.

To develop and validate the case review instrument to increase federal compliance with WIOA based performance indicators, answers to the following questions were pursued:

1. What are the WIOA performance indicators pertaining to case documentation?
2. Is the case review instrument developed reliable and valid?

First, the researcher reviewed existing materials including the RSA Monitoring and Technical Assistance Guide, existing case review instruments used by other agencies, and monitoring visit reports for other states – including interviews with directors.

Afterwards, a needs assessment was conducted. This included a survey to measure VR counselors’ level of understanding about the WIOA Performance Accountability Measures. In addition, a focus group was held to elicit VR supervisor and program director perceptions on quality documentation.

Next, researchers conducted case reviews on 10 cases to determine the availability of data necessary for the RSA reporting and monitoring.

These activities culminated in the development of the SCCB Case Review Instrument which featured four sections and 25 questions. The sections were as follows:

• Quality of Counseling Services and Documentation
• Quality of Vocational Preparation, Goal, Training, and Placement
• Quality of Closures
• Compliance with Regulations, Laws, and Policies

To test the newly developed Case Review Instrument three regional VR supervisors, the VR Consumer Services Director, and the current evaluator used it to review 20 cases. They were asked to rate the instrument using a Likert scale composed of four ratings: minimally/not complaint, mostly compliant, compliant, and not applicable.

A total of 97 cases were reviewed using the instrument. Twenty cases were reviewed by three regional VR supervisors, the VR Consumer Services Director, and the current evaluator. Reviewers were in 100% agreement in rating scales, and checked for inter-reliability. Seventy-seven cases were reviewed by the counselor trainer and the current evaluator. All cases reviewed were found to be either mostly compliant or compliant.

As a result of these findings, the case review instrument was adopted. The implementation of the case review instrument has created a path for development of training to better inform VR staff of compliance needs. The instrument is another tool for counselors, supervisors, and QA staff to target specific data collection areas and utilize the information for supervision and guidance. Case documentation procedures within the agency's system are being revised to reflect needs expressed by the counselors to encourage stronger case documentation. Increased use of the instrument will ultimately increase agency compliance with Federal performance indicators.

• Capstone Project Summary (PDF)
• Capstone Project Poster:  WORD | Powerpoint

More Capstone Projects: Capstone Project Hub.