Skip to main content

EQRO Activities

Hispanic Senior Couple Patient with Doctor
Top

PIPs Documents and Forms 

Validation of PIPs is one of the activities that the External Quality Review Organization (EQRO) conducts on an annual basis. Medicaid Managed Care Rules state that PIPs underway during previous 12 months be validated. The purpose of a PIP is to achieve, through ongoing measurements and interventions, significant improvement sustained over time in clinical or nonclinical areas.

The primary objective of PIP validation is to determine the MCO’s compliance with the requirements of 42 CFR §438.330(d)(2)(i-v), including:

  • Measurement of performance using objective quality indicators.
  • Implementation of systematic interventions to achieve improvement in quality.
  • Evaluation of the effectiveness of the interventions.
  • Planning and initiation of activities for increasing or sustaining improvement.

In its PIP evaluation and validation, Health Services Advisory Group (HSAG) used the Department of Health and Human Services, Centers for Medicare & Medicaid Services (CMS) publication, EQR Protocol 3: Validating Performance Improvement Projects (PIPs): A Mandatory Protocol for External Quality Review (EQR), Version 2.0, September 2012.1-1

HSAG evaluates the following components of the quality improvement (QI) process:

  1. The technical structure of the PIPs to ensure the MCOs designed, conducted, and reported PIPs using sound methodology consistent with the CMS protocol for conducting PIPs. HSAG’s review determined whether a PIP could reliably measure outcomes. Successful execution of this component ensures that reported PIP results are accurate and capable of measuring real and sustained improvement.
  2. The outcomes of the PIPs. Once designed, the MCO’s effectiveness in improving outcomes depends on the systematic identification of barriers and the subsequent development of relevant interventions. Evaluation of each PIP’s outcomes determined whether the MCO improved its rates through the implementation of effective processes (i.e., barrier analyses, intervention design, and evaluation of results) and, through these processes, achieved statistically significant improvement over the baseline rate. Once statistically significant improvement is achieved across all study indicators, HSAG evaluates whether the MCOs were successful in sustaining the improvement. The goal of HSAG’s PIP validation is to ensure that AHCA and key stakeholders can have confidence that reported improvement in study indicator outcomes is supported by statistically significant change and the MCOs’ improvement strategies.

The following are steps used by HSAG to validate each PIP:

Step I. Review the selected study topic(s)

Step II. Review the study question(s)

Step III. Review the identified study population

Step IV. Review selected study indicator(s)

Step V. Review sampling methodology (if sampling was used)

Step VI. Review the data collection procedures

Step VII. Review data analysis and interpretation of study results

Step VIII. Assess the improvement strategies

Step IX. Assess for real improvement the likelihood that reported improvement is “real” improvement

Step X. Assess for sustained improvement whether the documented improved has been sustained

HSAG’s PIP validation process is a desk review process; therefore, it is important for the MCOs to submit as much detail as possible at the time of submission.

For any questions related to the PIP process, please contact:

Christi Melendez, RN, CPHQ
Associate Executive Director
Performance Improvement Projects
Telephone: 602.801.6875
Email: cmelendez@hsag.com 


PIP Forms

FL PIP Completions Instructions
FL PIP Summary Form
PIP Validation Tool
Birth Outcomes PIP Methodology
Transportation PIP Methodology 
PPE PIP Methodology

1.1 Department of Health and Human Services, Centers for Medicare & Medicaid Services. EQR Protocol 3: Validating Performance Improvement Projects (PIPs): A Mandatory Protocol for External Quality Review (EQR), Version 2.0, September 2012. Available at: https://www.medicaid.gov/medicaid/quality-of-care/medicaid-managed-care/external-quality-review/index.html. Accessed on: May 2, 2018.

Performance Measure Validation (PMV) is one of the activities that the External Quality Review Organization (EQRO) conducts on an annual basis. Medicaid Managed Care Rules require validation of state-required performance measure data for managed care organizations (MCOs), prepaid inpatient health plans (PIHPs), prepaid ambulatory health plans (PAHPs), and primary care case management (PCCM) entities with financial incentives.

The purpose of the validation process is to:

  1. Evaluate the accuracy and validity of the performance measures reported by or calculated on behalf of the organizations;
  2. Determine the extent to which the performance measures were calculated according to specifications required by the state.

Health Services Advisory Group, Inc., (HSAG) conducts the validation activities as outlined in the Centers for Medicare & Medicaid Services (CMS) Performance Measure Validation Protocol. In some states, such as Florida, external auditors conduct HEDIS Compliance Audits™. Florida HMOs and PSNs submit their audited data and audit reports to the Agency for Health Care Administration (AHCA). The Florida Medicaid EQRO obtains the audit reports and final performance measure data from AHCA each year for the validation process. 

These are steps taken by HSAG to complete the validation process for HMOs and PSNs:

Step 1. Request information from MCOs

Step 2. Review MCO information and compliance audit reports

Step 3. Perform independent review of reported rates

Step 4. Aggregate and categorize validation findings


For questions related to the validation of performance measures process, please contact:

Mariyah Badani, JD, MBA, CHCA
Director: Audits, State and Corporate Services
Health Services Advisory Group, Inc.
602.801.6657
mbadani@hsag.com


HEDIS® is a registered trademark of the National Committee for Quality Assurance (NCQA)
HEDIS Compliance Audit™ is a trademark of the National Committee for Quality Assurance (NCQA)

Medicaid Managed Care Rules require that either the state, its designated agent, or the external quality review organization (EQRO) must conduct a review at least every three years to monitor compliance. The Agency for Health Care Administration (AHCA) has elected to perform compliance review activities for its managed care plans.

Currently, AHCA is developing a Compliance Review Plan. To date, AHCA has requested that HSAG assist in developing a Compliance Review Tool.

For questions related to the review of compliance, please contact:

Mary Wiley, BSW, RN, M.Ed.
Associate Executive Director, State & Corporate Services
602.801.6903
mwiley@hsag.com

Accurate and complete encounter data are critical to the success of any managed care program. State Medicaid agencies rely on the quality of encounter data submissions from its contracted health plans in order to monitor and improve the quality of care; establish performance measure rates; generate accurate and reliable reports; and obtain utilization and cost information. The completeness and accuracy of these data are essential in the state's overall management and oversight of its Medicaid managed care program and in demonstrating its responsibility and stewardship.

HSAG’s approach to evaluating encounter data for completeness and accuracy is consistent with the current Centers for Medicare & Medicaid Services (CMS) External Quality Review (EQR) Protocol 4, Validation of Encounter Data Reported by the MCO. In alignment with the protocols, HSAG incorporates one or more of the following core evaluation activities into its encounter data validation (EDV) studies:

  1. Information systems (IS) review—assessment of State’s and MCOs’ information systems and processes
  2. Administrative profile—analysis of State’s electronic encounter data completeness and accuracy
  3. Comparative analysis—analysis of State’s electronic encounter data completeness and accuracy through a comparative analysis between State’s electronic encounter data and the data extracted from the MCOs’ data systems
  4. Medical records review—analysis of State’s electronic encounter data completeness and accuracy through a comparative analysis between State’s electronic encounter data and the medical records

HSAG’s approach provides an effective way to identify and confirm the quality of encounter data collected and maintained by states while providing a clear roadmap toward improvement.

For questions related to the review of encounter data validation, please contact:

Amy Kearney
Director, Data Science and Advanced Analytics
602.801.6886
akearney@hsag.com

Focused studies can provide states with valuable information to assist them in evaluating managed care organization (MCO) performance on a variety of topics. Comparisons of MCO performance against specified standards, national benchmarks, or with each other will provide the Agency for Health Care Administration (AHCA) with information on which to base quality improvement efforts. 

Examples of focused studies Health Services Advisory Group (HSAG) have conducted are described below:

"Adolescent Well-Child Care" and "Identification of Special Health Care Needs." HSAG developed the study methodology, including the study questions, study indicators, data collection process, and analysis plan.
The “Identification of Special Health Care Needs” focused study relied entirely on administrative data obtained from specified state departments and the MCOs. The “Adolescent Well-Child Care” focused study involved data collection from medical records. MCOs were responsible for collecting the necessary medical records after HSAG identified the selected sample cases.
HSAG will continue to conduct focused studies as requested by AHCA. 

For questions related to the review of focused studies, please contact:

Mary Wiley, BSW, RN, M.Ed.
Associate Executive Director, State & Corporate Services
602.801.6903
mwiley@hsag.com

Key information related to all external quality review (EQR) activities must be made available to all interested parties, including managed care organizations, various divisions within the Agency for Health Care Administration, and key stakeholders. In order to meet this objective, Health Services Advisory Group (HSAG) uses several mediums to communicate with the key groups, including a public website, a secured File Transfer Protocol (FTP) site for the exchange of sensitive data or information, electronic mail, conference calls, quarterlymeetings with key stakeholders, quarterly meetings, and written communications.

In addition, HSAG has established internal mechanisms to continually evaluate the effectiveness of each approach and ensure that information is being communicated in the most effective, timely, and direct manner possible. 

HSAG also provides educational opportunities during quarterly meetings, the answers to frequently asked questions on the website, and links to other key resources related to EQR services. 

For questions related to the dissemination and education activities, please contact:

Mary Wiley, BSW, RN, M.Ed.
Associate Executive Director, State & Corporate Services
602.801.6903
mwiley@hsag.com

State Medicaid agencies are required by the Social Security Act, section 1932(c), to provide for an annual external, independent evaluation of quality outcomes and the timeliness of, and access to, services provided by the state’s Medicaid managed care organizations (MCOs). Medicaid Managed Care Rules require state Medicaid agencies to provide an annual report of this evaluation using information from mandatory quality review activities. 

Health Services Advisory Group (HSAG) compiles an annual technical report each year that includes the following information for each quality review activity conducted:

Objectives
Technical methods of data collection and analysis
Descriptions of the data obtained
Conclusions from the data, including improvement opportunities.
The report includes an assessment of the MCOs' strengths and weaknesses with respect to quality, access, and timeliness. 

The Florida Agency for Health Care Administration’s annual technical reports can be viewed on the Resources page of this website. 

For questions related to the technical report, please contact:

Mary Wiley, BSW, RN, M.Ed.
Associate Executive Director, State & Corporate Services
602.801.6903
mwiley@hsag.com