Use Case 3: Greater granularity in PQRS published benchmarks

From Demand-Driven Open Data for HHS
Jump to: navigation, search


Use case summary


  • Need for greater granularity in PQRS published benchmarks. The benchmarks are currently derived from the aggregation of quality data submitted through multiple data collection methods, but it would be helpful if in addition the data from each collection method were used to calculate separate benchmarks for that collection method (e.g. set of benchmarks for data collected through GPRO registry, set of benchmarks for data collected through individual EP registry, CMS claims, etc.).
  • It is also helpful to understand, which providers are absent. In other words, for providers who are eligible for MU which ones didn't submit


  • Value to customer: Will be able to provide greater and more actionable insights to the healthcare providers.
  • Value to industry/public: The value of this is to enable additional insights into the relative data quality and measure performance of different data collection methods, thereby improving our understanding of how to more accurately and effectively measure care delivery and patient safety risks.
  • Reduces providers’ administrative costs.
  • Leaves more time and resources to patient care
  • Impact: Decide which method of reporting to choose


  • SA Ignite

Current data and limitation


  • Fields:
    • Measure name
    • PQRS #
    • NQF # (opt)
    • CMS eMeasure # (opt)
    • mean
    • standard dev
    • performance year
  • Format:
    • CSV format via download or API, rather than PDF
  • Update frequency: Annual
  • Lag time:
    • Make avail in Q2, rather than November. Because can’t do much in Nov for prior year. (PQRS measures due 3/31)
  • Joins between datasets: N/A


Acknowledge limitation

  • The requested lag time is to be available Q2 (rather than November) for prior year results.
    • Answer from Data Owners (at CMS/CCSQ): Reducing the lag time is not possible under the current conditions, unless the data processing methods are significantly amended.
      • Currently there are budgetary, contractual and logistical limitations.
      • The data analysis and production of this report depends on an external contractor, with a predefined schedule identified in the contract.
      • Even if the contract could be amended, there's a lot of work involved in the processing that isn't easily sped up. Much of the calculation is still done manually, depending on the collection method. Data analysis takes the entire time allotted after close of submission period. Additionally, acceptable deviation and confidence levels need to be met. Also, different reporting groups might have different factors for calculating weighted means.

Short term workaround

  • Try to issue FOIA requests for this data.
    • Approach: In order to speed delivery and improve consistency, specify the system of record, query to run, and individuals or contractors who currently run it
    • Additional challenges:
      • Acceptable deviation and confidence levels for each reporting method need to be met.
      • There may also be technical issues, as different reporting groups might have different factors for calculating weighted means. Much of the calculation is done manually. For example, this is the case for data based on patient records
    • Next steps:
      • CMS/CSSQ needs to investigate how much PQRS data exists at this level of granularity.
      • CMS/CCSQ needs to first gauge comfort within the organization to work on a different PQRS reporting method, as well as reviewing the stated regulatory authority for publishing this data. Depending on its statutory authority, CMS may not be able to be proactive about publishing this data.
      • However, limitation of statutory authority doesn't preclude this information from being released via FOIA request. GC (general counsel) indicated there’s no regulatory authority to release the MU data, while the FOI office indicated that it falls under FOIA and is releasable.

Long term implementation

  • Eventually, Physician Compare would be a destination that could hold this data, although not likely in the format requested here
    • Commitment: At the moment there are no committed release dates
    • Current state: Now Physician Compare includes badges on PQRS, but no benchmarks. Additionally, the data can’t be downloaded and accessed via machine readable API
    • Challenge and observation (identified by CMS/CCSQ):
      • Committing to providing this data via a consumer application, rather than machine readable format, may be easier to commit to from a policy perspective.
      • CMS feels that this data is better provided directly from CMS, rather than 3rd party consumer apps, because it's important for consumers to have confidence and trust in the data. The fear is that 3rd party apps would obscure the original source of the data, thereby stripping it of its original provenance. For similar reasons, implementing an API might be more likely than allowing a download of the entire database.