A Pilot Study to Develop VR Case Review Instrument for WIOA Performance Measure Data Collection

  • Developed by Elaine Robertson, South Carolina Commission for the Blind

Takeaways for VR Counselors & Administrators

    • Implementation of a consistent case review instrument is recommended to increase WIOA compliance rates and provide a better overall picture of agency performance.
    • Modification of case management software may be necessary to gather required data. Software training and a desk reference guide can increase accuracy of data entered.
    • Targeted case reviews should be completed monthly for specific measures and the review instrument itself should be examined annually to ensure all RSA changes are implemented as needed.
    • Researchers suggest that agencies retain dedicated Quality Assurance personnel for case review purposes.
    • The review instrument can be used by supervisors to prepare annual performance reviews with counselors.

    Background

    In 2015 the Rehabilitation Services Administration implemented the Workforce Innovation and Opportunity Act (WIOA). This legislation changed the way Vocational Rehabilitation (VR) services were provided in all state agencies, and also introduced new performance measures the RSA would use to verify compliance. The act shifted VR focus from quantity to quality, requiring a major change in how data was collected, interpreted, and reported.

    This project focused on developing an instrument that would incorporate performance measures into case documentation; provide a consistent method for interpreting quality; and that would gauge compliance with WIOA, state, and internal agency regulations.

    Methods

    To develop and validate the case review instrument to increase federal compliance with WIOA based performance indicators, answers to the following questions were pursued:

    1. What are the WIOA performance indicators pertaining to case documentation?
    2. Is the case review instrument developed reliable and valid?

    First, the researcher reviewed existing materials including the RSA Monitoring and Technical Assistance Guide, existing case review instruments used by other agencies, and monitoring visit reports for other states – including interviews with directors.

    Afterwards, a needs assessment was conducted. This included a survey to measure VR counselors’ level of understanding about the WIOA Performance Accountability Measures. In addition, a focus group was held to elicit VR supervisor and program director perceptions on quality documentation.

    Next, researchers conducted case reviews on 10 cases to determine the availability of data necessary for the RSA reporting and monitoring.

    These activities culminated in the development of the SCCB Case Review Instrument which featured four sections and 25 questions. The sections were as follows:

    • Quality of Counseling Services and Documentation
    • Quality of Vocational Preparation, Goal, Training, and Placement
    • Quality of Closures
    • Compliance with Regulations, Laws, and Policies

      Validation Study

      To test the newly developed Case Review Instrument three regional VR supervisors, the VR Consumer Services Director, and the current evaluator used it to review 20 cases. They were asked to rate the instrument using a Likert scale composed of four ratings: minimally/not complaint, mostly compliant, compliant, and not applicable.

      Results

      A total of 97 cases were reviewed using the instrument. Twenty cases were reviewed by three regional VR supervisors, the VR Consumer Services Director, and the current evaluator. Reviewers were in 100% agreement in rating scales, and checked for inter-reliability. Seventy-seven cases were reviewed by the counselor trainer and the current evaluator. All cases reviewed were found to be either mostly compliant or compliant.

      Implication for Practice or Future Research

      As a result of these findings, the case review instrument was adopted. The implementation of the case review instrument has created a path for development of training to better inform VR staff of compliance needs. The instrument is another tool for counselors, supervisors, and QA staff to target specific data collection areas and utilize the information for supervision and guidance. Case documentation procedures within the agency’s system are being revised to reflect needs expressed by the counselors to encourage stronger case documentation. Increased use of the instrument will ultimately increase agency compliance with Federal performance indicators.

        Resources

          Capstone projects

          More Capstone Projects: Capstone Project Hub.

          Leave a Reply

          Your email address will not be published. Required fields are marked *