Secretarial Review and Publication of the Annual Report to Congress Submitted by the Contracted Consensus-Based Entity Regarding Performance Measurement

AGENCY: Office of the Secretary of Health and Human Services, HHS.

ACTION:

Notice.

SUMMARY: This notice acknowledges the Secretary of the Department of Health and Human Services' (HHS) receipt and review of the Annual Report submitted to the Secretary and Congress by the contracted consensus-based entity (CBE) as mandated by section 1890(b)(5) of the Social Security Act, as created by section 183 of the Medicare Improvements for Patients and Providers Act of 2008 (MIPPA) and amended by section 3014 of the Affordable Care Act of 2010. The statute requires the Secretary to review and publish the report in the Federal Register together with any comments of the Secretary on the report not later than six months after receiving the report. This notice fulfills those requirements.

FOR FURTHER INFORMATION CONTACT:

Ann Page (202) 260-6473.

I. Background

Rising health care costs coupled with the growing concern over the level of and variation in quality and efficiency in the provision of health care raise important challenges for the United States. Section 183 of MIPPA created Section 1890 of the Social Security Act, which requires the Secretary of the Department of Health and Human Services (HHS) to contract with a consensus-based entity to perform multiple duties pertaining to health care performance measurement. These activities support HHS's efforts to promote high-quality, patient-centered, and financially sustainable health care. The statute mandates that the contract be competitively awarded for a period of four years and may be renewed under a subsequent bidding process.

In January, 2009, a competitive contract was awarded by HHS to the National Quality Forum (NQF) for a four-year period. The contract specified that the CBE should conduct its business in an open and transparent manner, provide the opportunity for public comment and ensure that membership fees do not pose a barrier to participation in the scope of HHS's contract activities, if applicable.

The HHS four-year contract includes the following major tasks:

Priority Setting Process: Formulation of a National Strategy and Priorities for Health Care Performance— The CBE shall synthesize evidence and convene key stakeholders to make recommendations on an integrated national strategy and priorities for health care performance measurement in all applicable settings. The CBE shall give priority to measures that: Address the health care provided to patients with prevalent, high-cost chronic diseases; provide the greatest potential for improving quality, efficiency and patient-centered health care; and may be implemented rapidly due to existing evidence, standards of care or other reasons. Additionally, the CBE shall take into account measures that: May assist consumers and patients in making informed health care decisions; address health disparities across groups and areas; and address the continuum of care across multiple providers, practitioners and settings.

Endorsement of Measures: Implementation of a Consensus Process for Endorsement of Health Care Quality Measures— The CBE shall provide for the endorsement of standardized health care performance measures. This process shall consider whether measures are evidence-based, reliable, valid, verifiable, relevant to enhanced health outcomes, actionable at the caregiver level, feasible to collect and report, and responsive to variations in patient characteristics such as health status, language capabilities, race or ethnicity, and income level and is consistent across types of health care providers including hospitals and physicians.

Maintenance of Consensus Endorsed Measures—The CBE shall establish and implement a process to ensure that endorsed measures are updated (or retired if obsolete) as new evidence is developed.

Promotion of the Development of Electronic Health Records— The CBE shall promote the development and use of electronic health records that contain the functionality for automated collection, aggregation, and transmission of performance measurement information. However, in January of 2013, this task was repealed and, as a result, removed from the CBE's statutory duties by the American Taxpayer Relief Act (Pub. L. 112-240, Title VI, § 609(a)(2)).

Convening Multi-Stakeholder Groups— The CBE shall convene multi-stakeholder groups to provide input into the selection of certain categories of quality and efficiency measures, including measures for use in certain specific Medicare programs, for use in programs that report performance information to the public, and for use in health care programs that are not included under the Social Security Act. The multi-stakeholder groups consider measures to be implemented through the federal rulemaking process for various federal health care quality reporting and quality improvement programs including those that address certain Medicare services provided through hospices, hospital inpatient and outpatient facilities, physician offices, cancer hospitals, end stage renal disease (ESRD) facilities, inpatient rehabilitation facilities, long-term care hospitals, and psychiatric hospitals and home health care programs.

Annual Report to Congress and the Secretary— Under section 1890(b)(5)(A) of the Act, by not later than March 1 of each year (beginning with 2009) the CBE shall submit to Congress and the Secretary of HHS an annual report. The report shall contain a description of:

(i) The implementation of quality and efficiency measurement initiatives and the coordination of such initiatives with quality and efficiency initiatives implemented by other payers;

(ii) recommendations on an integrated national strategy and priorities for health care performance measurement;

(iii) performance of its duties required under its contract with HHS;

(iv) gaps in endorsed quality and efficiency measures, which shall include measures that are within priority areas identified by the Secretary under the National Quality Strategy established under section 399HH of the Public Health Service Act (National Quality Strategy), and where quality and efficiency measures are unavailable or inadequate to identify or address such gaps;

(v) areas in which evidence is insufficient to support endorsement of quality and efficiency measures in priority areas identified by the Secretary under the National Quality Strategy, and where targeted research may address such gaps; and

(vi) the convening of multi-stakeholder groups to provide input on: (1) The selection of quality and efficiency measures from among such measures that have been endorsed by the CBE and such measures that have not been considered for endorsement by the CBE but are used or proposed to be used by the Secretary for the collection or reporting of quality and efficiency measures; and (2) national priorities for improvement in population health and the delivery of health care services for consideration under the National Quality Strategy.

Section 1890(b)(5)(B) of the Social Security Act requires Secretarial review and publication of this report in the Federal Register, together with any comments of the Secretary on the report not later than 6 months after receiving the report.

The first annual report covered the performance period of January 14, 2009 to February 28, 2009 or the first six weeks post contract award. In March 2009, NQF submitted the first annual report to Congress and the Secretary of HHS. Given the short timeframe between award and the statutory requirement for the submission of the first annual report, this first report provided a brief summary of future plans. The Secretary published a notice in the Federal Register in compliance with the statutory mandate for review and publication of the annual report on September 10, 2009 (74 FR 46594).

In March 2010, NQF submitted to Congress and the Secretary the second annual report covering the period of performance of March 1, 2009 through February 28, 2010. The second annual report was published in the Federal Register on October 22, 2010 (75 FR 65340) to comply with the statutorily required Secretarial review and publication.

In March 2011, NQF submitted the third annual report to Congress and Secretary of HHS. The third annual report, which covers March 1, 2010 through February 28, 2011, was published in the Federal Register on September 7, 2011 (76 FR 55474).

In March 2012, NQF submitted its fourth annual report to Congress and the Secretary. The report covers the period of performance of January 14, 2011 through January 13, 2012. The fourth annual report was published in the Federal Register on September 14, 2012 (77 FR 56920).

In March 2013, NQF submitted its fifth annual report to Congress and the Secretary. The report covers the period of performance of January 14, 2012 through December 31, 2012. Because the first annual report covered only six weeks, there have been five annual reports under this four-year contract. This notice complies with the statutory requirement for Secretarial review and publication of the fifth NQF annual report.

II. March 2013—Consensus-Based Entity Report to Congress and the HHS Secretary

Submitted in March 2013, the fifth annual report to Congress and the Secretary spans the period of January 14, 2012 through December 31, 2012.

A copy of NQF's submission of the March 2013 annual report to Congress and the Secretary of HHS can be found at: http://www.qualityforum.org/Publications/2013/03/2013_NQF_Report_to_Congress.aspx. The fifth NQF annual report is reproduced in section III of this notice.

III. NQF Report of 2012 Activities to Congress and the Secretary of the Department of Health and Human Services

This report was funded by the U.S. Department of Health and Human Services under contract number: HHSM-500-2009-00010C.

1. Executive Summary

In the last six years, Congress passed statutes that call upon HHS to work with a consensus-based entity (the entity) to facilitate multi-stakeholder input into (1) setting national priorities for improvement in quality and (2) recommending use of performance measures in federal programs to achieve these priorities. The statutes also call upon a consensus-based entity to review and endorse a portfolio of standardized performance measures to be used by stakeholders in public and private quality improvement and accountability programs. Note: The relevant statutory language appears in italicized text throughout this report. The first of these statutes is the 2008 Medicare Improvements for Patients and Providers Act (MIPPA) (PL 110-275), which established the responsibilities of the consensus-based entity by creating section 1890 of the Social Security Act and was passed under President Bush. The second statute is the 2010 Patient Protection and Affordable Care Act (ACA) (Pub. L. 111-148), which modified and added to the consensus-based entity's responsibilities, and was passed under President Obama. The 2013 American Taxpayer Relief Act (Pub. L. 112-240) extended funding under the MIPPA statute to the consensus-based entity through fiscal year 2013. HHS awarded contracts related to the consensus-based entity identified in the statute to the National Quality Forum (NQF). As amended by the above laws, the Social Security Act (the Act)—specifically section 1890(b)(5(A))—also mandates that the entity report to Congress and the Secretary of the Department of Health and Human Services (HHS) no later than March 1st of each year. The report must include descriptions of: (1) How NQF has implemented quality and efficiency measurement initiatives under the Act and coordinated these initiatives with those implemented by other payers; (2) NQF's recommendations with respect to activities conducted under the Act on an integrated national strategy and priorities for healthcare performance measurement in all applicable settings; (3) NQF's performance of the duties required under its contract with HHS; (4) gaps in endorsed measures that NQF has identified, including measures that are within priority areas identified by the Secretary under HHS' national strategy; (5) areas NQF has identified in which evidence is insufficient to support endorsement of measures in priority areas identified by the National Quality Strategy, and where targeted research may address such gaps, and (6) the matters described in clauses (i) and (ii) of paragraph (7)(A) of section 1890(b). To address the last item, the report will cover the new multi-stakeholder group input duties for the consensus-based entity as outlined in section 3014(a), which created section 1890(b)(7) and (8) of the Act. The first of these duties includes providing multi-stakeholder input on the selection of quality and efficiency measures both endorsed and those not endorsed by the entity, that are used or proposed to be used by the Secretary for collection or reporting of quality and efficiency measures. The second duty requires that the consensus-based entity provide multi-stakeholder group input on national priorities for improvement in population health and in the delivery of healthcare services for consideration under the National Quality Strategy.

This fourth Annual Report highlights NQF's work conducted between January 14, 2012 and December 31, 2012 related to these statutes and conducted under a federal contract with the U.S. Department of Health and Human Services.The deliverables produced under contract in 2012 are referenced throughout this report, and a full list is included in Appendix A.

Facilitating Coordinated Action To Achieve the National Quality Strategy

Section 1890(b)(1) of the Social Security Act mandates that the entity shall synthesize evidence and convene key stakeholders to make recommendations on an integrated national strategy and priorities for healthcare performance measurement in all applicable settings. In making such recommendations, the entity shall ensure that priority is given to measures: that address the health care provided to patients with prevalent, high-cost, chronic diseases; that focus on the greatest potential for improving the quality, efficiency, and patient-centeredness of healthcare; and that may be implemented rapidly due to existing evidence and standards of care. In addition, the entity will take into account measures: that may assist consumers and patients in making informed healthcare decisions; address health disparities across groups and areas; and address the continuum of care a patient receives, including services furnished by multiple healthcare providers or practitioners and across multiple settings.

Under section 1890(b)(5)(A)(ii) of the Social Security Act, the entity is mandated to include in the annual report a description of the recommendations it has made, with respect to activities conducted under the Social Security Act, on an integrated national strategy, and priorities for healthcare performance measurement in all applicable settings.

Since 2009, the NQF-convened National Priorities Partnership (NPP) has helped to provide multi-stakeholder input into the selection of high-impact goals, related priorities, and subsequent strategies that constitute the first-ever National Strategy for Quality Improvement in Healthcare (NQS). Released in 2011, the NQS outlines three specific aims for the U.S. healthcare system—better care, healthy people and communities, and affordable care. To achieve these aims, the NQS established six priorities to help the healthcare community focus their efforts, including:

• Making care safer by reducing harm caused in the delivery of care;
• Ensuring that each person and family are engaged as partners in their care;
• Promoting effective communication and coordination of care;
• Promoting the most effective prevention and treatment practices for the leading causes of mortality, starting with cardiovascular disease;
• Working with communities to promote wide use of best practices to enable healthy living; and
• Making quality care more affordable for individuals, families, employers, and governments by developing and spreading new healthcare delivery models.

The NPP is a collaborative public-private partnership of more than 50 organizations that have a shared stake in how healthcare is delivered, received, and paid for. NPP continues to advise HHS on how to evolve the NQS' three aims, and its counsel was well reflected in HHS's 2012 National Strategy for Quality Improvement in Healthcare, an annual NQS progress report required by Congress.

Beyond forging agreement at the strategic goal level, it is challenging to get leaders to implement agreed-upon strategies at the care delivery and community level, given limited time and resources. In 2012, NPP focused on how to advance patient safety by aligning its work with HHS' “Partnership for Patients” effort. Through a series of web-based and in-person meetings that NPP hosted throughout 2012, nearly 2,700 participants from multiple sectors were able to learn about and share new improvement approaches, information, tools, and professional connections to accelerate their individual contributions to achieving safety related improvements. At a more detailed level, NPP developed action plans to focus a range of national and local organizations in diverse sectors on how to align efforts to reduce preventable readmissions and improve maternity care, relying on proven interventions. NPP also created a web-based system or “action registry” to track related commitments to improvement activities focused on readmissions and maternity care to enable learning across participants. Launched in the fourth quarter of 2012, the registry now houses over 50 actions by 30 different organizations.

Endorsing and Maintaining Measures, Related Tools, and Information

Under section 1890(b)(2) of the Social Security Act, the entity must provide for the endorsement of standardized healthcare performance measures. As part of the endorsement process, NQF is required to consider whether measures are evidence-based, reliable, valid, verifiable, relevant to enhanced health outcomes, actionable at the caregiver level, feasible for collecting and reporting data, responsive to variations in patient characteristics, and consistent across healthcare providers. In addition, under section 1890(b)(3), the NQF must maintain endorsed measures, by establishing and implementing a process to ensure that endorsed measures are retired if obsolete or brought up to date as new evidence is developed.

NQF strategically manages its portfolio of 700-plus endorsed measures to increase impact and decrease burden, growing the portfolio in some areas and shrinking it in others. More specifically, it replaces existing measures with those that are better, reflect new medical evidence, or are more relevant; removes measures that are no longer effective or where the evidence base has evolved; and expands the portfolio to address well-recognized measurement gaps.

The NQS priorities guide the management of the measure portfolio by NQF expert committees. In addition to concentrating on endorsing measures suitable for public reporting, performance-based payment, and other accountability purposes, NQF evolves its portfolio so that the measures are also clinically relevant and actionable for providers. Payers and patients are interested in measures that they can use to compare and select providers; clinicians and hospitals seek clinically relevant measures to benchmark themselves against so they have the information they need to focus their improvement efforts for the benefit of their patients. A mix of measures is essential to creating and continuously evolving a portfolio that meets the needs of diverse stakeholders.

In 2012, NQF completed 16 endorsement projects—reviewing 430 submitted measures and endorsing 301 measures, or 70 percent. This set included 81 new measures and 220 measures that maintained their endorsement after being considered in light of new evidence and/or against new competing measures submitted to NQF for consideration. The newly endorsed measures align with needs identified in the NQS and address several critical areas, including patient outcomes, underserved populations, healthcare disparities, and hospital readmissions.

In comparison, NQF completed 11 projects and endorsed 170 measures in 2011. This increased productivity can be attributed to efforts to make the review process more efficient—the average measure review time decreased from 12 months to 7 months during 2012—as well as to other enhancements to the endorsement process. Specifically, as part of the Consensus Development Process pilot program, NQF provided earlier, more detailed feedback to measure developers about a first-order criterion (i.e., importance to measure) to further the goal that development dollars are spent on measures that are viewed as consequential by the field. Furthermore, when a measure is re-evaluated for continued endorsement, NQF now requires committees to consider the measure's use and whether such use has resulted in improvement or has led to unintended consequences, ensuring that committee members are informed about the measure's impact.

Under section 1890(b)(4) of the Social Security Act, the entity has been responsible for promoting the development and use of electronic health records (EHRs) that contain the functionality for automated collection, aggregation, and transmission of performance measurement information.

In an effort to move beyond measures that rely on administrative data or that are collected from paper-based medical records, NQF continued its work in 2012 to facilitate the development and reporting of electronic measures, or eMeasures, that can help accelerate the adoption of electronic health records (EHRs). Such efforts include work at the granular level (e.g., standardizing data elements so they can be collected from varied EHRs to build eMeasures) and at the more conceptual level (e.g., the NQF-convened eMeasure Learning Collaborative). Created by NQF at the behest of measure developers, EHR vendors, HHS, and clinicians, the eMeasure Learning Collaborative is a forum for sharing best practices and tackling issues that are barriers to developing and implementing eMeasures, such as figuring out how to enhance “upstream” communication between measure developers and other stakeholders so that affected parties have the opportunity to collaborate on data requested and its representation in eMeasure logic during the measure development process. In 2012, NQF also launched the Health IT Knowledge Base and glossary to facilitate a unified understanding of terms and measurement approaches used in EHRs and more broadly, health IT, and to disseminate best practices, among other projects.

Aligning Accountability Measures To Enhance Value

Under section 1890(b)(1) of the Social Security Act, the entity shall synthesize evidence and convene key stakeholders to make recommendations and priorities for healthcare performance measurement in all applicable settings.

Under section 1890(b)(5)(A)(i) of the Social Security Act, the entity must report on the implementation of quality and efficiency measurement initiatives under the Social Security Act and the coordination of these initiatives with quality and efficiency initiatives implemented by other payers.

Under section 1890(b)(7) of the Social Security Act, NQF is specifically responsible for convening multi-stakeholder groups to provide input to the Secretary of HHS on the selection of certain categories of NQF-endorsed and non-endorsed quality and efficiency measures (measures NQF has not considered for endorsement but the Secretary uses or is proposing to use for the collection or reporting of quality and efficiency measures). Beginning in 2012, NQF has been required to transmit the input of the multi-stakeholder groups to the Secretary not later than February 1st of each year. Under section 1890(a)(5), the Secretary must consider multi-stakeholder input as part of a pre-rulemaking process the Secretary must complete prior to the adoption of measures during the Federal rulemaking process. NQF provides this multi-stakeholder input through its Measure Applications Partnership (MAP).

Agreement about how to define quality, safety, and costs in a portfolio of endorsed measures is an important first step toward measure alignment, which then needs to be followed by consensus across stakeholder groups about the use of endorsed measures.

The NQF-convened MAP—which comprises stakeholders from a wide array of healthcare sectors and 10 federal agencies, as well as 110 subject matter experts—focuses on recommending measures for federal public reporting, payment, and other programs to enhance healthcare value. As part of its mission, MAP also strives for alignment with the private sector on the use of such measures. In February 2012, MAP provided multi-stakeholder input to HHS about the considered use of measures in over 17 different federal Medicare benefit programs and the Electronic Health Record (EHR) Incentive Program as a part of its first annual pre-rulemaking report required by statute. This input was well-heeded, as evidenced by a degree of concordance—or agreement between MAP's recommendations and the Centers for Medicare & Medicaid Services (CMS) final rules for quality reporting, public reporting, and value-based purchasing programs issued in 2012—which averaged 70 percent concordance across programs.¹ Where discordance exists, it appears to be due to timing. For example, in some cases, such as the Physician Quality Reporting System (PQRS), CMS is moving measures rapidly into a program to encourage clinician participation and concurrently encouraging that these measures be reviewed by NQF for possible endorsement.

To help guide future measure development related to the NQS and to inform use of measures in value-based programs going forward (including future annual pre-rulemaking reports to HHS), MAP released a Strategic Plan for Measurement in October 2012. A key part of the plan focuses on defining the concept of “families of measures” in high-impact areas, some of which cross conditions and settings. The objective of these families, or sets of measures, is to knit together related measures currently found in different programs, care settings, levels of analysis, and populations to drive improvement and reduce measurement burden. In addition, the plan calls for further engagement of stakeholders to glean additional feedback about measure use and usefulness.

At the same time, MAP released its Families of Measures report, which defines measure families in four key areas—safety, care coordination, cardiovascular, and diabetes care—with the goal of promoting more cohesion and integration of care regardless of setting, provider, level of intensity, or timing. An additional and equally important goal is reducing measurement and reporting burden through alignment for hospitals, physicians, and other providers as it relates to these four areas.

A 2012 NQF analysis (conducted outside of the federal contract) of NQF-endorsed measures in use shows that about 29 percent of measures are being used by two or more key stakeholders simultaneously, including the federal government, private payers, states, communities, and other users. Given its size and reach, the federal government is an important driver, using more than half of NQF's measure portfolio in its various pay-for-reporting and pay-for-performance programs, followed by private payers and states using 41 percent and 28 percent, respectively. Further, NQF's analysis shows that alignment in use of the same measures increased across these key sectors between 2011 and 2012.^{2 3} A 2011 RAND study of 75 organizations revealed a strong preference for NQF-endorsed measures where they exist because they are vetted, evidence-based, and known to be more credible with providers.⁴

Filling Measurement Gaps

Under section 1890(b)(5)(A)(iv) of the Social Security Act, the entity is required to report on gaps in endorsed quality and efficiency measures including measures within priority areas identified by HHS under the agency's National Quality Strategy, and where quality and efficiency measures are unavailable or inadequate to identify or address such gaps. Under section 1890(b)(5)(v) of the Social Security Act, NQF is also required to report on areas in which evidence is insufficient to support endorsement of quality and efficiency measures in priority areas identified by the Secretary under the National Quality Strategy and where targeted research may address such gaps.

The science of performance measurement continues to evolve in response to the needs and preferences of various stakeholders, new and updated data platforms, the capacity of providers to collect and report measures, and other factors. In 2012, NQF conducted an extensive analysis of its current measures portfolio against both the National Quality Strategy priority areas and high-impact conditions to meet requirements under section 1890(b)(5)(A)(iv) of the Social Security Act. This analysis provides a more in-depth understanding of what NQF-endorsed measures exist against key strategic frameworks, which of these measures are being used in the field, and where gaps persist—either because the measures have not yet been developed or they are in existence but are not being used.

The extent to which each NQS priority at the goal level has NQF-endorsed measures available to drive change is varied but generally promising. For example, a large part (40%) of the NQF portfolio addresses the important area of patient safety which includes healthcare acquired conditions and hospital readmissions. Fewer measures (7 percent) address patient and family engagement. Overall, measures for specific goals—including shared decision-making, patient navigation and self-management, shared accountability, healthy lifestyle behaviors, community interventions to improve health, and access, cost, and resource use—are less prevalent.

Looking across both the NQS priority areas and high-impact Medicare and child health conditions, the analysis found gaps in measures of preventive care, patient-reported outcomes (particularly quality of life and functional status), appropriateness (particularly for specialty care), access to timely palliative care, and health and healthcare disparities. Additionally, the analysis revealed the need for better population-level measures to assess improvements in health and healthcare. An assessment of the NQF portfolio of endorsed measures revealed that while certain high-impact conditions have an abundance of measures—e.g., cardiovascular disease, end-stage renal disease, and diabetes—many of the high-impact childhood conditions have few or no NQF-endorsed measures. Finally, all but one of the 92 NQF-endorsed measures in use in federal and at least two other non-federal programs address a specific NQS goal or a high-impact condition.

While certainly there is room for improvement, the analysis suggests that the existing portfolio generally addresses agreed upon frameworks and that there is alignment in use of such measures across various sectors. Going forward, resources should be dedicated to delving more deeply into the identified gap areas to prioritize measure development and endorsement efforts so that the most needed measurement gaps are addressed first.

Furthermore, NQF's efforts are focused on furthering alignment as it relates to measurement strategies to enhance healthcare value through its public-private partnerships and its evidence-based, consensus-driven method for reviewing and endorsing measures. Ultimately, however, for the U.S. healthcare system to be transformed, measurement-driven efforts will need to be mutually reinforced with changes to current payment and delivery systems that drive the system toward greater integration and accountability. Only then will we be able to put the U.S. healthcare system on the path to achieving the NQS' three, interconnected, and ambitious aims.

2. Facilitating Coordinated Action To Achieve the National Quality Strategy

Section 1890(b)(1) of the Social Security Act mandates that the entity shall synthesize evidence and convene key stakeholders to make recommendations on an integrated national strategy and priorities for healthcare performance measurement in all applicable settings. In making such recommendations, the entity shall ensure that priority is given to measures: That address the healthcare provided to patients with prevalent, high-cost chronic diseases; that have the greatest potential for improving the quality, efficiency, and patient-centeredness of healthcare; and that may be implemented rapidly due to existing evidence and standards of care. In addition, the entity will take into account measures that may assist consumers and patients in making informed healthcare decisions, address health disparities across groups and areas, and address the continuum of care a patient receives, including services furnished by multiple healthcare providers or practitioners and across multiple settings.

The National Quality Strategy (NQS), released in March 2011, set forth a cohesive roadmap for achieving patient-centered, affordable care that promotes healthy people and communities (see pages 3-4 for a more detailed explanation). Upon its release, its authors emphasized that the national quality strategy requires the active engagement and support of healthcare stakeholders across the country for quality improvements and success.

For the increasing number of stakeholders that have committed to making the NQS a reality, the path and methods to achieve its aims are not always apparent. Additionally, as the hard work of achieving care of the highest value accelerates, stakeholders are increasingly recognizing that performance measurement and quality improvement are only achievable by working across sectors and organizations, and they seek effective and efficient ways to connect across the healthcare delivery system.

The NPP focused its 2012 efforts on bringing diverse people and organizations together in their pursuit of the NQS, and in conducting analyses and activities that helped to refine the next critical priorities of the healthcare community.

Advising on the National Quality Strategy

NPP members called for the creation of the NQS and in 2012 continued to shape its direction by offering input to the HHS Secretary. In September 2011, HHS asked the NPP to recommend measures for evaluating progress in achieving the NQS. This input was integrated into the 2012 National Strategy for Quality Improvement in Healthcare, an annual NQS progress report required by Congress. The progress report reflected near-universal agreement with NPP recommendations. Multi-stakeholder input into the NQS and follow-on work to achieve its goals embody the spirit of alignment encouraged by the NQS authors, ensuring that the strategy is informed, embraced, and viewed as achievable by both public and private sectors. Without this shared vision, progress is likely to be marred by competing, unfocused, or discordant efforts.

Identifying and Spreading Solutions To Achieve the National Quality Strategy

Under section 1890(b)(5)(A)(i) of the Social Security Act, the entity is to provide a description of its implementation of quality and efficiency measurement initiatives under the Social Security Act and the coordination of those initiatives with those implemented by other payers.

In addition to offering multi-stakeholder input on the NQS, the NPP focused on helping to disseminate proven and scalable solutions for its implementation; making connections across sectors and between organizations; and inspiring people to take highly focused, coordinated, and targeted action. Much of this work happened as part of the HHS Partnership for Patients patient safety effort, which has two ambitious and important goals: reducing hospital-acquired conditions by 40 percent and preventable hospital readmissions by 20 percent by the end of 2013.

Establishing the “who, what, how, and when” of action is the first step in solving large-scale challenges that cut across organizations and sectors. To that end, NPP partners and an extended network of contributors (more than 750 in total) spent part of 2012 developing these problem-solving pathways—with an initial focus on fashioning shared solutions to improving maternity care and reducing preventable readmissions. The NPP selected these two areas for specific reasons. Current trends in maternity care and readmissions demonstrate an opportunity for improvement that can simultaneously reduce unnecessary patient harm and healthcare costs. Both areas also represent aspects of healthcare ripe for pooling and focusing the efforts of many—patients and families, providers, payers, and policymakers, to name a few.

For example, since 1979, the American Congress of Obstetricians and Gynecologists (ACOG) has advocated for the avoidance of elective deliveries before 39 completed weeks gestation, yet early elective inductions are common in the United States despite the known potential harms for mothers and babies.⁵ Similarly, rates of cesarean section have risen in recent decades to nearly 32 percent despite potential harms, including greater likelihood of asthma for the child. In fact, the cesarean rate is rising fastest among women who are least likely to benefit—healthy women at low risk of labor and birth complications.⁶ Studies reveal that higher cesarean rates do not lead to improved outcomes, and rates above 15 percent may do more harm than good.⁷ Furthermore, there is strong evidence to support the need to address avoidable admissions and readmissions. Almost one in five Medicare patients discharged from the hospital is readmitted within 30 days, putting patients at increased risk of complications or infections and accounting for approximately $15 billion of excess Medicare spending each year.^{8 9 10} While some admissions and readmissions are planned and appropriate, approximately 40 percent of hospital admissions among nursing home residents may be avoidable.¹¹

In addition to these two specific areas of focus, NPP hosted several larger scale forums on behalf of the Partnership for Patients in 2012. NPP-hosted forums were designed to identify innovative ways to help multiple organizations meet Partnership for Patients' safety goals and to help spread proven patient safety interventions. Without these exchanges, organizations often find themselves trying to improve in a vacuum, working with a limited number of ideas and/or interventions, or struggling to innovate given their human and financial resources. The structure of these forums, oriented around idea exchanges and sharing of case studies and examples, fostered efficient information sharing, so that those on the frontlines of improving patient safety were supported in their efforts and therefore could more readily effect change. More than 400 organizations that support the Partnership for Patients attended these events. The first three meetings were focused on education regarding the National Quality Strategy and the importance of alignment between sectors; catalyzing action; and sharing success stories in achieving patient safety. The November 2012 NPP-Partnership for Patients event focused exclusively on how to achieve meaningful patient and family engagement, which is essential for solving all patient safety issues and achieving a patient-centered healthcare system. After the first meeting in January 2012, 100 percent of attendees felt the meeting enhanced their ability to contribute to public-private sector collaboration. NPP augmented the four in-person forums with online educational `webinars.' In total, over the course of 2012, nearly 2,700 people from multiple sectors participated in NQF-hosted webinars and in-person events in support of the Partnership for Patients.

In 2012, NQF designed a web-based, interactive “registry” where organizations can share information about their own actions to advance the NQS; search data about the actions of others; find partners to work with; and learn from others. The registry, available on the NQF Web site, allowed for broader engagement, participation, and content that facilitates alignment around a focused set of patient safety activities and that clarifies who is doing what, when, with whom, and to what end. Launched in the fourth quarter of 2012, the registry now houses over 50 actions by 30 different organizations.

Deliverables Associated With These Activities

3. Supporting National Healthcare Measurement Needs

Under section 1890(b)(2) of the Social Security Act, the entity must provide for the endorsement of standardized healthcare performance measures. The endorsement process shall consider whether measures are evidence-based, reliable, valid, verifiable, relevant to enhanced health outcomes, actionable at the caregiver level, feasible for collecting and reporting data, responsive to variations in patient characteristics, and consistent across healthcare providers. In addition, under section 1890(b)(3) of the Social Security Act, the NQF must maintain endorsed measures, including retiring obsolete measures and bringing other measures up to date.

Standardized healthcare performance measures help clinicians understand whether the care they offered their patients was optimal and appropriate, and if not, where to focus their efforts to improve the care they deliver. Measures are also used by all types of public and private payers for a variety of accountability purposes, including feedback and benchmarking, public reporting, and incentive-based payment. Lastly, measures are an essential part of making healthcare more transparent to all, important for those who receive care or help make care decisions for loved ones.

Working with a variety of stakeholders to build consensus, NQF reviews and endorses healthcare performance measures that underpin federal and private-sector initiatives focused on enhancing the value of healthcare services.

Ten years ago, NQF endorsed its first voluntary, national consensus performance measures to answer the call for standardized measurement of healthcare services. These first measures were a stepping-stone for creating a consensus-driven effort that bridged nearly every interested party in healthcare. The 10-year result of this national experiment is a portfolio of more than 700 NQF-endorsed measures, most of which are in use; a more information-rich healthcare system; and a substantial emerging body of knowledge about measure development, use, and quality improvement.

In the past five years, NQF, working in partnership with HHS and others, has focused more intensely on measures that add value and reduce burden for those who provide, pay for, and receive care. This movement has been facilitated through more stringent evaluation criteria that place greater emphasis on evidence and a clear link to outcomes, demonstrable impact and gaps in care, and testing that demonstrates measures' reliability and validity. NQF also has laid the foundation for the next generation of measures, including guidance on composite measurement, patient-reported outcome measures, disparities-sensitive measures, electronic or eMeasures, and measures that evaluate complex but important areas such as resource use and population health. These activities are intended to inform the path toward targeted, prioritized measure development.

There is increasing evidence that NQF's stringent criteria, portfolio management strategies, and collaboration with developers are having the desired effect on the portfolio. For example, in 2012 we observed the following:

• Guidance that expressed NQF's strong preference for outcome measures and that required process measures to demonstrate a clear link to outcomes led to more endorsed outcome measures. At the end of 2012, 27 percent of the measures in NQF's portfolio were outcome measures, compared to 24 and 18 percent in 2011 and 2010, respectively.
• A focus on harmonization resulted in fewer duplicative measures, and steering committees selecting the best-in-class measure whenever possible.
• Developers submitted more tested measures—which are more reliable, valid, and likely to meet NQF endorsement criteria—given NQF's increased emphasis on requirements for measure testing. With fewer untested measures to evaluate, steering committees were able to focus more on evaluating “better” measures.

To apply the concept of constant improvement to its own work, NQF conducted in 2012 Lean improvement activities and other initiatives and/or projects intended to make the consensus development process more predictable, efficient, and navigable for those who develop and evaluate measures, while still maintaining the rigor of its multi-stakeholder process. Measure developers primarily seek an earlier window to get broad-based committee input on a measure concept they are considering investing in; those who use measures are interested in process changes that may further shrink review cycle time while maintaining rigor. All parties are focused on ways to make sure finite measure development resources are used to meet the greatest measurement needs.

To address these issues, NQF took steps to explore restructuring of its Consensus Development Process (CDP) in order to provide early guidance to measure developers on whether a measure concept meets NQF's criterion for “importance to measure and report” before they invest time and resources to fully develop and test a measure. The results of the pilot project, often referred to as the “two-stage CDP,” will be available in 2013; results will be used to drive additional enhancements that meet the critical needs of measure developers.

NQF worked to enhance its approach to harmonization, specifically helping those who review measures to more consistently and adeptly recognize an opportunity for aligning measures. In 2012, NQF also conducted work to help committees evaluate measures for usability, a criterion for NQF endorsement with which steering committee members often struggle during deliberations.

Lastly, outside of the HHS process improvement activities around measure development, NQF created a new multi-stakeholder task force on consensus, which, working with NQF staff, led a series of focus groups and research exercises to determine a definition of consensus and how to establish consensus in rare instances when the NQF membership vote is split.

Results of NQF's Lean improvement work included reducing the average measure endorsement cycle time from 12 to 7 months, which is an important milestone to ensuring that the measures that matter most to our changing healthcare system are available for use as quickly as possible all without sacrificing the rigor of the endorsement process. Other results included the development of standard work for staff, developers, and committee members. This task force on consensus is slated to produce findings in early 2013.

Current State of NQF Measures Portfolio: Constricting and Expanding To Meet Evolving Needs

NQF's measure portfolio includes more than 700 performance measures, covering a variety of different conditions and care settings. The portfolio is carefully managed in a variety of ways. First, working with various expert committees, NQF removes or puts into “reserve status” measures that consistently perform at the highest levels or “top out.” This step signals an improvement success and helps to ensure that time is spent instead measuring areas in need of improvement. Second, NQF works with those who create measures to “harmonize” related or near-identical measures to eliminate nuanced differences. Harmonization is critical to reducing measurement burden for providers, who have been inundated with various misaligned measurement requests. Successful harmonization may result in fewer endorsed measures for providers to report and for payers and consumers to interpret. Lastly, where appropriate, NQF works with measure developers to replace multiple process measures with more meaningful outcome metrics. In 2012, NQF removed 103 measures from its portfolio for a variety of reasons: Measures no longer met endorsement criteria; measures were harmonized with other similar, competing measures; or measure developers chose to retire measures they no longer wished to maintain.

While NQF pursues these proven trimming strategies to make its measure portfolio appropriately lean, it also aggressively seeks measures from the field that will help to fill known measure gaps and to align with the NQS goals. Several important factors motivate NQF to expand its portfolio, including: (1) The need for eMeasures; (2) pressure for measures that are applicable to multiple clinical specialties and settings of care; (3) national pursuit of new payment models such as bundled payment; and (4) the need for more advanced measures that help close cross-cutting gaps, such as care coordination and patient-reported outcomes. The measure portfolio reflects the combined “dynamic yet static” effect of these strategies: Although the portfolio is constantly changing due to new measures cycling in and others cycling out, the relative number of endorsed measures remained steady in 2012. Specifically, 93 measures were added and 103 measures were removed from the portfolio.

The table below provides a snapshot of how the current NQF-endorsed measure portfolio aligns with the NQS, with the percentages reflecting the proportion of NQF-endorsed measures that support each of the six priorities. Some measures are counted in multiple priority areas. The table shows gaps in emerging measurement areas, including affordability, patient- and family-centered care, and community health and individual well-being. Work conducted in 2012 helped to close these known measure gaps and to pave the way for innovative measure development by the healthcare field.

Furthermore, seven measure developers account for 64 percent of NQF's portfolio:

Specific Measure Endorsement Accomplishments

In 2012, NQF completed 16 measure endorsement projects—reviewing 430 submitted measures and endorsing 301. These endorsed measures include 81 new measures and 220 measures that NQF expert committees concluded could maintain their previous endorsement after being reviewed against NQF's criteria and compared to new evidence or competing measures. Overall, measures undergoing maintenance were endorsed at a rate of 55 percent, and new measures submitted for endorsement were endorsed at a rate of 89 percent.

Case in point: In the last year clinical projects with a large number of process measures had markedly lower endorsement rates for maintenance measures (e.g., perinatal care, 44 percent; pulmonary, 44 percent; and renal disease, 36 percent). Newer measurement areas that are highly valued by clinicians and patients had higher endorsement rates, including disparities measures at 75 percent and palliative care at 64 percent. The disparities measures were primarily outcome measures, while the palliative measures were primarily process measures.

The measures endorsed by NQF in 2012 align with needs called out in the NQS and address several critical areas including patient outcomes, hospital readmissions, underserved populations, and healthcare disparities. A complete listing on measures and measurement frameworks endorsed by NQF in 2012 under contract with HHS is available in Appendix A. Highlights include the following:

Patient-reported experience measures. The healthcare community is working toward a more patient-driven system, in which individual needs and preferences are incorporated into care decisions. Measures that address patient experience, coupled with clinical measures, allow for a more comprehensive view of patient care. For example, coupling a measure that assesses whether post-surgical instructions for care were clear to the patient and his or her caregiver with measures that assess hip surgery complication rates creates a more complete picture of a patient's experience.

In 2012, NQF endorsed several measures addressing patient experience in various care settings. For example, a measure from the American College of Surgeons evaluates patient satisfaction during hospitalization for surgical procedures. A measure from the Agency for Healthcare Research and Quality focuses on effective provider communication with patients regarding disease management, medication adherence, and test results. The American Medical Association developed seven measures that were endorsed; these measures address concerns such as individual health literacy, availability of language services, and patient engagement with providers in clinician offices and acute care facilities. Finally, measures from the Center for Gerontology and Health Care Research and the PROMISE Center evaluate how bereaved family members perceive the quality of care provided to loved ones in hospices, nursing home facilities, and hospitals.

NQF also convened two expert workshops to explore how patient-reported outcomes (PROs) can be effectively used in performance measurement. Defined as a patient's health status as reported by the patient, PROs are seen as the next step forward in building a patient-centered healthcare system. In the surgical example, a PRO might be information gleaned from a patient about when she could resume basic activities of daily living, start exercising, or return to work. The NQF portfolio already contains some patient-reported outcome measures. For example, patient reports are the basis of an NQF-endorsed measure of depression remission six months after treatment developed by Minnesota Community Measurement. Experiences by community coalitions, physician practices, and others implementing PROs helped inform NQF expert committees over the past year as they figured out how to overcome data, reporting, and methodological barriers to developing and using PRO-based performance measures.

Readmissions measures. About one in five Medicare beneficiaries who leaves a hospital is readmitted within 30 days. Such unplanned readmissions—many of which are potentially preventable—take a significant toll on patients and their families, often resulting in prolonged illness or pain, emotional distress, and days of lost work. These readmissions also cost Medicare about $15 billion annually.¹² Although Medicare beneficiaries are more likely to be rehospitalized, the private sector also spends billions of dollars each year on patients who have an unplanned readmission to the hospital within a month of an initial stay.

NQF endorsed two hospital-wide, all-cause readmission measures and three condition-specific readmission measures that can help the healthcare community better understand and appropriately reduce hospital readmission rates. These measures align with major safety and affordability issues. However, as performance measures are increasingly used in pay-for-performance programs, concerns about the potential for unintended consequences, such as a negative impact on providers that care for vulnerable populations, have increased. These issues were prominent considerations during the 2012 endorsement deliberations over the hospital-wide, all-cause readmission measure (NQF measure #1789), which was ultimately endorsed. To address multiple stakeholders' needs and concerns about the newly endorsed readmissions measures, the NQF Board of Directors issued guidance regarding the use of hospital-wide measures as it ratified the measure:

Multiple factors affect readmission rates and other measures including the complexity of the medical condition and associated therapies; effectiveness of inpatient treatment and care transitions; patient understanding of and adherence to treatment plans; patient health literacy and language barriers; and the availability and quality of post-acute and community-based services, particularly for patients with low incomes. Readmission measurement should reinforce national efforts to focus all stakeholders' attention and collaboration on this important issue.

In response to continued concerns about the use of the new hospital-wide, all-cause readmission measure (#1789), NQF proposed a series of steps to take place after endorsement of that particular measure, including monitoring implementation; employing an expert multi-stakeholder group to review “dry run” data provided by CMS regarding measure #1789; evaluating new readmission measures for new conditions; and establishing ongoing monitoring approaches that ensure that more systematic feedback from measure users is integrated into endorsement deliberations. NQF also reviewed updates to the readmission measures to remove planned readmissions from the condition-specific measures that are generally not considered signals of quality, and is continuing efforts to harmonize hospital and health plan all-cause readmission measures.

Patient safety measures. Americans are exposed to more preventable medical errors than patients in other industrialized nations, costing the United States close to $29 billion per year in additional healthcare expenses, lost worker productivity, and disability.¹³ These costs are passed on in a number of ways, including higher insurance premiums and taxes and lost wages. Proactively addressing medical errors and unsafe care will help to protect patients from harm, lead to more effective and equitable care, and appropriately reduce costs.

NQF endorsed 32 patient safety measures in 2012, focusing on complications such as healthcare-associated infections, falls, medication safety, and pressure ulcers. These measures closely align with goals of the Partnership for Patients to make care safer.

Resource use measures. Healthcare expenditures in the United States are unmatched by any other country. This spending, however, has not resulted in better health for Americans. In general, the United States lags behind other countries in terms of mortality, patient satisfaction, access to care, or quality of care within the healthcare system.^{14 15 16} Patients, insurers, state and regional leaders, federal policymakers, employers, and providers are all attuned to affordability and increasingly focused on how we can measure and reduce healthcare expenditures without harming patients.

NQF endorsed its first set of resource use measures—designed to understand how healthcare resources are being used—in January 2012, and it endorsed an additional set in April 2012. These measures will offer a more complete picture of what drives healthcare costs from several perspectives. For example, one endorsed measure evaluates a primary care provider's risk-adjusted frequency and intensity of all services used to manage patients—including inpatient/outpatient, pharmacy, laboratory, radiology, and behavioral health services—using standardized prices. Another measure evaluates a primary care provider's risk-adjusted cost effectiveness at managing his patient population using actual prices paid by health plans. Similar measures also evaluate total resources used by individual patients with specific conditions, such as asthma and chronic obstructive pulmonary disease, over the course of a measurement year. And other measures evaluate total costs over an episode of care, such as costs associated with hip/knee replacement, from diagnosis to treatment to rehabilitation. Used in concert with quality measures, these resource use measures will enable stakeholders to identify opportunities for creating a higher value healthcare system.

Harmonized behavioral health measures. In 2012, NQF endorsed 10 measures related to mental health and substance abuse, including measures of treatment for individuals experiencing alcohol or drug dependent episodes; diabetes and cardiovascular health screening for people with schizophrenia or bipolar disorder; and post-care follow-up rates for hospitalized individuals with mental illness. As a part of this process, NQF also brought together CMS and NCQA to harmonize two related measures into one measure addressing antipsychotic medication adherence in patients with schizophrenia.

A multiple chronic conditions measurement framework. People with multiple chronic conditions (MCCs) now comprise more than 25 percent of the U.S. population^{17 18} and this number is expected to grow. This population is more likely to see multiple clinicians, take five or more medications, and receive care that is fragmented, incomplete, inefficient, and ineffective.^{19 20 21 22 23} They are at significantly higher risk of adverse outcomes and complications.

Despite the growing prevalence of people with MCCs, existing quality measures typically do not address issues associated with the care for individuals with MCCs, largely because of data sharing challenges and because measures are typically limited to addressing a singular disease and/or specific setting. As a result, NQF endorsed a measurement framework that establishes a shared vision for effectively measuring the quality of care for individuals with MCCs. Measure developers can use this framework to more quickly create measures for this population, filling a current measurement gap.

Healthcare disparities measures. Research from the Institute of Medicine shows that racial and ethnic minorities often receive lower quality care than their white counterparts, even after controlling for factors such as insurance coverage, socioeconomic status, and comorbidities.²⁴ Such disparities are exacerbated by additional factors, including that racial and ethnic minorities have poorer health status in general, face more barriers to care, and are more likely to have poor health literacy.

With funding from the Robert Wood Johnson Foundation, NQF established a more detailed picture of how to approach measurement of healthcare disparities across settings and populations, beginning with a commissioned paper outlining methodological concerns. To ensure that disparities in care can be addressed most effectively, NQF developed an approach to identify measures that are more sensitive to disparities and, as such, should be stratified. From there, NQF endorsed 12 performance measures that focused on patient-provider communication, cultural competence, and language services, among other issues. Now that these measures are endorsed, HHS has more opportunity to include these kinds of measures, which address a key NQS measurement priority, in federal programs.

Streamlining Measure Information

Various healthcare entities gather, store, and need to access information about performance measures. Over the years, different measure information systems have been built, each with differing purposes, structure, and content. This diversity of places and approaches to storing such information confounds the ability to find and coordinate pieces of information about a given measure, such as a specific version, unique identifying number or name, specifications, purpose and context, and benchmarking results.

HHS asked NQF to use its role as a neutral convener to work with a variety of public- and private-sector organizations to conduct a “Registry Needs Assessment.” The assessment was geared toward understanding how various stakeholders currently approach gathering and storing performance measure information; assessing the desirability of a different approach including but not limited to a single “measure registry” system; and identifying the barriers to achieving more aligned and definitive ways to store and access consistent and comprehensive information about measures. The findings included recommendations for first steps such as developing shared definitions of measure “metadata” and versioning standards to enable alignment of measure information.

The Global to the Granular: NQF's Role in Accelerating the Adoption of eMeasures

Under section 1890(b)(4) of the Social Security Act, the entity was tasked with promoting the development and use of electronic health records that contain the functionality for automated collection, aggregation, and transmission of performance measurement information.

Currently, healthcare data largely live within system silos and on paper rather than in electronic form, which makes it nearly impossible for data to follow patients through various settings in which they receive care. Healthcare is safer and better coordinated when electronic health records (EHRs) and other clinical information technology systems reliably capture and share data across providers and patients to facilitate care—and as a byproduct of the clinical process—generate performance measurement information. Wide adoption of this kind of electronic infrastructure will spur implementation of the NQS, but has been hampered by a variety of issues.

NQF's health IT work in 2012 focused on pulling together disparate organizations that play a role in moving quality from a paper-based world to one facilitated by technology. The faster we reach consensus on approaches to this new world, the faster we may achieve the goal of a fully empowered and connected electronic information system designed with the patient in mind.

At the global level, NQF launched a series of activities designed to promote shared understanding among those involved in advancing electronic measurement and data infrastructure. It convened the eMeasure Learning Collaborative, a new environment for promoting best practices related to development and implementation of measures applied to electronic data sources (i.e., eMeasures). eMeasures are an innovation in advancing quality measurement, but significant barriers hamper their wider scale creation, adoption, and use. Through two in-person meetings and other virtual convenings, NQF brought together hundreds of stakeholders including government representatives, EHR vendors, measure developers, clinicians, and hospitals—creating a unique forum for these parties to work together on new eMeasurement approaches.

Specific eMeasure best practices emerged from this Learning Collaborative, particularly in three areas: Organizational leadership, data representation and clinical workflow, and learning health systems. For example, regarding data representation, all participants identified the need for measure developers and other stakeholders to communicate earlier in the eMeasurement process, particularly when measure developers are selecting data and representing data in eMeasure logic. For this best practice to become a reality, a national structure and process must exist to enable this level of dialogue. With respect to organizational leadership, participants suggested that provider organizations create inter-professional, physician-led teams focused on an integrated approach to eMeasure adoption, including data capture, reporting, workflow, clinical decision support, and evidence-based practice.

Several of NQF's 2012 projects sought to facilitate a unified understanding of terms and measurement approaches used in the health IT field, so that measure developers and implementers, health IT vendors, standards organizations, and other users of eMeasures and tools work with a similar lexicon. For example, NQF launched the Health IT Knowledge Base, providing answers to some of the most common technical questions about NQF's related initiatives. Since August 2012, NQF added more than 70 new entries to the frequently asked questions section, stemming from its interactions with eMeasure users and developers. NQF also added a glossary with more than 150 terms and definitions. As a complement to the Knowledge Base, NQF provided opportunities for stakeholders to learn about best practices in eMeasurement through a series of NQF-hosted health IT webinars that reached more than 1,400 people during the past 12 months.

As quality measurement shifts to an electronic platform, additional clarity is needed regarding the testing that assures that eMeasures can be used for a range of accountability applications, which require both precision and reliable and valid results. NQF worked with CMS and the Office of the National Coordinator for Health Information Technology (ONC) to ensure that the data capture for eMeasures is feasible without impeding clinical workflow. NQF's health IT initiatives in 2012 scaled down to the granular level as well, to help standardize the efforts of the creators and users of eMeasures. Developed by NQF, the Quality Data Model (QDM) is an “information model” that defines concepts used in quality measures and clinical care in a way that allows the information to be collected automatically from data already stored in an EHR.

An example illustrates how the QDM can simplify and standardize the electronic collection and reporting of quality measures. If a physician's office wants to use its EHR to report on a measure that assesses the percentage of patients with a diagnosis of coronary artery disease (CAD) who were prescribed a lipid-lowering therapy, the EHR must first identify the patients with CAD within the physician's practice and then determine whether the patients had the therapy. If the physician's performance is going to be compared to her peers, then her EHR must define these elements in exactly the same way as every other EHR. The QDM supports this type of query regardless of the type of EHR by defining the necessary standard data elements (e.g., active diagnosis, active medication administered/ordered/dispensed) and the type of coding that the EHR may use to express the result (e.g., ICD-9 code for diagnosis; RxNorm for medication, etc.). When all measure specifications are written in a common way, EHR vendors can more easily ensure that their EHRs can support quality measurement, and the validity of electronic-based reporting programs will likely increase. NQF released an updated version of the QDM in December 2012, which focused on simplifying and standardizing QDM measure logic to support implementation of the federal Meaningful Use regulations. NQF also regularly receives ongoing feedback and insights into best practices from a User Group of measure developers, physicians, hospitals, and EHR vendors who are currently actively involved in eMeasure use.

NQF's work in standardizing eMeasurement extends to measure development. NQF partnered with a software developer to develop the Measure Authoring Tool (MAT), which is a publicly available, free, web-based tool designed to allow measure developers to create eMeasures using the aforementioned QDM, without needing to write programming code. At the end of 2012, NQF prepared to transition the day-to-day operation of the MAT to HHS, giving HHS the opportunity to better position the MAT and eMeasures in federal programs using EHR-based performance measurement, and to support the MAT's evolution.

Also in 2012, NQF completed the Critical Paths for Creating Data Platforms project. This effort helped assess the readiness of electronic data to support innovative measurement concepts and recommended steps to address data and infrastructure gaps and barriers in two high-priority domains: care coordination and patient safety. The care coordination report focused on transitions of care and communication of the patient plan of care. The patient safety report focused on effective use of infusion devices (e.g., giving medication through an IV) in acute care settings. The ability to capture data across settings is fundamental to gauging, for example, the degree of care coordination in a healthcare system. The final reports from these projects delineated specific steps that the government and private sector can take to enable electronic measurement in these areas.

4. Aligning Measure Use To Enhance Value

Under section 1890(b)(5)(A)(i) of the Social Security Act, the entity is required to provide a description of its implementation of quality and efficiency measurement initiatives under the Social Security Act and the coordination of those initiatives with those implemented by other payers.

Under section 1890A of the Social Security Act, HHS is required to establish a pre-rulemaking process under which a consensus-based entity (currently NQF) would convene multi-stakeholder groups to provide input to the Secretary on the selection of quality and efficiency measures for use in federal programs as specified under section 1890(b)(7)(B) of the Social Security Act. The list of quality and efficiency measures HHS is considering for selection will be publicly published no later than December 1 of each year. No later than February 1 of each year, NQF will report the input of the multi-stakeholder groups which will be considered by HHS in the selection of quality and efficiency measures for use in federal programs as specified under section 1890(b)(7)(B) of the Social Security Act.

Alignment with respect to use of the same performance measures is a critical strategy for accelerating improvement, reducing wasteful reporting burden, and enhancing transparency in healthcare. The NQF-convened Measure Applications Partnership (MAP), launched in the spring of 2011 as mandated by the Patient Protection and Affordable Care Act (Pub. L. 111-148, section 3014), is a key facilitator of measure alignment across federal programs and between the public and private sectors. The input that the MAP provides to HHS for purposes of the pre-rulemaking process and national priorities under the National Quality Strategy results from multiple stakeholders composed of representatives from more than 60 major private-sector stakeholder organizations, 10 federal agencies, and 40 individual technical experts MAP's input enhances HHS's ability to coordinate its quality and efficiency measurement initiatives with those initiatives implemented by other payers.

More specifically, MAP provides a forum for annual multi-stakeholder input into which performance measures are used in federal public reporting and pay-for-performance programs in advance of related regulations being issued. This approach augments traditional rulemaking, allowing the opportunity for substantive dialogue with HHS before rules are issued, a chance for alignment across programs with respect to use of measures, and consideration of longer term implications. MAP also provides a unique forum for public- and private-sector leaders to develop and then broadly vet a future-focused performance measurement strategy (outlined in the MAP strategic plan below), as well as the shorter term recommendations for that strategy on an annual basis in pre-rulemaking reports. MAP strives to offer recommendations that are cross-cutting and coordinated across: settings of care; federal, state, and private programs; levels of measurement analysis; payer type; and points in time.

Published on February 1, 2012, MAP's first pre-rulemaking report offered recommendations related to 17 federal programs.²⁵ This report:

• Recommended that 40 percent of the measures that CMS proposed at the end of 2011 move into federal programs targeting clinicians, hospitals, and post-acute care/long-term care (PAC/LTC) settings via rules issued in 2012, with another 15 percent targeted for future consideration after further development, testing, and feasibility issues are worked out. MAP did not support inclusion of the remaining 45 percent primarily because many of the measures did not have enough information, specificity, testing, or proof of implementation feasibility to guide MAP measure evaluation and selection. See Appendix C for the criteria MAP used to guide measure selection.
• Expressed clear preference for both using NQF-endorsed measures and for developing more robust feedback loops. Over 90 percent of the measures that MAP supported for inclusion in the first round of pre-rulemaking input were currently NQF-endorsed, with the remainder likely eligible for expedited review. In addition to these criteria, NQF is establishing more robust feedback loops that can help HHS, MAP, and the broader field to discern which of the endorsed measures are best suited for inclusion in future reporting and value-based purchasing programs. More specifically, in 2012 MAP analyzed what internal and external sources exist to obtain feedback from end users and informally engaged MAP members to understand how they would prioritize varying types of feedback information.²⁶
• Considered how to further align measures across public programs and with the private sector with the goal of more targeted, inter-related sets of measures that are reported by different kinds of providers, in different settings, and across time.
• Laid out guiding principles for a three- to five-year measurement strategy where priority is placed on: (1) Measures that drive the system toward meeting the NQS; (2) measures that are person- rather than clinician-focused; and (3) measures that span settings, time, and types of clinicians. Person-centered measurement provides information about what matters to patients (e.g., “Will I be able to run after I recover from knee surgery?”) and that is specific to patient populations or care over time, (e.g., “Did I get the care and support needed to manage my diabetes so that I did not lose my vision or my mobility?”). This kind of measurement is predicated on a redesigned delivery and payment system and an HIT-enabled environment that facilitates both coordination and integration of care for a range of patients across the continuum.

Federal Medicare and Meaningful Use rules issued over the course of 2012 largely followed the MAP pre-rulemaking recommendations for inclusion or exclusion of measures in over 20 different payment and reporting programs that MAP was asked to consider. However, concordance between the HHS final rules issued in 2012 with the MAP 2012 recommendations varied depending on the program (see table below for key programs). Over 70% concordance was observed for the majority of relevant programs. Of the two programs that had lower concordance with MAP Recommendations, there were only five measures in one program (ESRD QIP) relevant to the analysis, and there was a relatively short time period available for HHS to consider MAP's input for the other program (Meaningful Use). There were various reasons for the individual instances of discordance. Where CMS did not finalize measures that MAP supported, the most common issue was difficulty of data collection or other burden imposed by those measures. Excluded from the concordance analysis were many measures that had not yet been reviewed or endorsed by NQF at the time of MAP's evaluation, leaving MAP with insufficient information to provide a definitive “Support” or “Do Not Support” recommendation. For example, in the Medicare Physician Fee Schedule rule, CMS included a number of non-endorsed measures that address the broad array of medical specialties to engage more physicians in federal physician-level programs. Going forward NQF is poised to quickly move these measures through review for potential endorsement.

MAP Strategic Plan for Measurement. To spur progress toward a defined set of goals and priorities related to the NQS—which include improved quality and safety, more transparency, and enhanced value—MAP developed a three-year strategic plan for measurement (2012-2015). This plan was released on October 1, 2012, and is intended to inform HHS's future measure development planning, as well as shape annual rulemaking advice in the years ahead. The plan has the following three major components:

• Define sets of measures as families of measures with the objective of knitting together related measures currently found in different programs, care settings, levels of analysis, and populations. This approach complements the program-specific recommendations that MAP made in its pre-rulemaking report. Individual measures are carefully selected to work together as a “family” to drive the overall system toward better performance in a given area, promote more patient-centeredness, and decrease reporting burden for providers. Families of measures are linked to a high-impact condition (e.g., diabetes) or an NQS priority (e.g., safety) and are intended to promote further measure alignment by specifying within the families more discrete core measure sets focused on hospitals, clinicians, or post-acute/long-term care. See MAP's Families of Measures report or for a summary of the report, see page 28.
• Engage stakeholders that develop, report, and use measures to glean feedback about the use and usefulness of measures. The idea is to create more effective two-way communication so that the experiences of end users directly inform MAP's recommendations to HHS, contribute to the thinking of the diverse stakeholders that participate directly and indirectly in MAP's activities, as well as inform the work of measure developers as they address identified measurement gaps in a more coordinated fashion.
• Develop analytic support for MAP decision making. The goal is to further enrich MAP's thinking and decision-making by integrating important data and information that are developed across NQF as a strategic byproduct of its different activities. These include input to priority setting and strategies, measurement review and endorsement, and advice on measure selection. This function would also draw upon the various outside efforts under way to glean information about measure use and impact. The analysis and integration of internal and external data will inform and likely refine MAP's overall selection criteria, as well as its recommendations to HHS in future pre-rulemaking reports. In addition, an independent third-party evaluation is planned to determine whether MAP is meeting its overall objectives.

The MAP pre-rulemaking recommendations and strategic plan largely reflect the current reality of our siloed healthcare payment and delivery systems, but anticipate a future system with shared accountability for patient welfare, community health, and stewardship of scarce resources.

Families of Measures

MAP selected safety, care coordination, cardiovascular conditions, and diabetes as its first focus areas for identification of families of measures— all areas called out in the NQS and/or leading causes of mortality. MAP's first families of measures report was published on October 1, 2012.

MAP reviewed 676 measures across these 4 topics, using criteria laid out in the report as a guide to inform selection. Of these measures, MAP recommended 55 safety, 60 care coordination, 37 cardiovascular, and 13 diabetes measures for inclusion in 4 distinct families of measures. MAP further defined more discrete core measures, which include available measures, and gaps specific to a care setting (e.g. hospitals, post-acute care/long-term care), level of analysis (e.g. individual clinicians), or population drawn from each family of measures and made program-specific recommendations in its 2013 pre-rulemaking report. MAP anticipates identifying families of measures for patient and family engagement, population health, affordability/cost, and mental health in 2013, pending funding decisions.

MAP defined families of measures with the intent that their implementation would lead to performance improvement and further cohesion and synergy of care in a targeted area. Measures in a given family bridge healthcare settings, types of providers, and time and are interconnected in the way patients would ideally like to experience care. Families of measures also include identifying measure gaps, which strongly signal to developers where new measures are needed, and can help facilitate prioritization of funding for measure development.

For example, the safety family of measures contains 9 topic areas and 22 subtopic areas. The topic areas include but are not limited to reducing healthcare-acquired infections and obstetrical adverse events and increasing procedural safety. Examples of specific gaps in the safety family of measures include post-discharge follow-up of infections in ambulatory settings, ventilator-associated events with special considerations for the pediatric population, and infection measures reported as rates rather than ratios, which would be more meaningful to consumers. The 55 measures selected for the safety family of measures follow themes such as creating a culture of safety, patient and caregiver engagement, reporting meaningful safety information, and cost of care implications. These measures were selected for their ability to cross settings to simultaneously affect patients, caregivers, and purchasers and to ultimately increase safety for all patients.

Measure Use and Alignment

Although the advantages of measure alignment are many, few studies have systematically examined this phenomenon. A 2011 RAND study of 75 diverse organizations found that nearly all used NQF-endorsed measures, although there was considerable variability in which measures were used and for what purposes. Most used NQF-endorsed measures in quality improvement programs, followed closely by use in public reporting and then payment programs. The 2011 study also found that the organizations surveyed indicated a strong preference for NQF-endorsed measures where they exist because they are vetted, evidence-based, and known to be more credible with providers.²⁷

In 2011 and 2012, NQF conducted initial research outside of the HHS contract to better understand which organizations are using NQF-endorsed measures and where there is alignment across sectors with respect to that use.^{28 29} In addition, NQF is developing more systematic approaches to capturing detailed feedback from end users about the usefulness of NQF measures in driving improvements in health and healthcare.

The 2012 analysis showed that 86 percent of the 706 NQF-endorsed measures were in use, with the balance of the portfolio not in use largely consisting of measures recently endorsed (last 1-3 years) and expected to be used in the near future. Federal use of the NQF portfolio was stable at about 50 percent. Private payer use of the NQF portfolio grew from 21 percent to 35 percent during this period; state use grew from 21 percent to 23 percent. Much of the increase in private payer use is likely attributable to better data collection by NQF, rather than increased use of NQF-endorsed measures by private payers.

The federal government, private plans, and states appear to be increasingly using the same NQF-endorsed measures. In 2012, the federal government and private payers used the same 76 measures in accountability programs, or 13 percent of the 606 NQF-endorsed measures in use. During the same period, federal and state alignment was 48 measures, or 8 percent, and private payer and state alignment was 51 measures, or 8 percent. In 2012, 25 measures were simultaneously used by the federal government, private payers, and states. When all users are taken into account (including local communities, registries and others users), about 29 percent of the NQF-endorsed portfolio was used by two or more stakeholders in 2012.

NQF Facilitates National, State, and Local Measure Alignment

• Improvement Targets: Inform the National Quality Strategy (National Priorities Partnership)
• Measures: Endorse and harmonize measures
• Incentives: Advise HHS on reporting/payment programs (Measure Applications Partnership)
• National-Local Actions: Develop tools to align use of measures (Quality Positioning System or QPS) and efforts of national/local organizations implementing strategies at the delivery system level (National Priorities Partnership)

Alignment at the Community Level

Given the number and diversity of community-based efforts, it is challenging to get a comprehensive sense of how standardized measures are being used at the local, state, or regional levels. That said, the number of regional multi-stakeholder collaboratives or alliances that are collecting, reporting, and in some cases paying on the basis of performance measures appears to have grown over the past number of years. As of October 2012, the Robert Wood Johnson Foundation has cataloged on its Web site a compendium of nearly 260 state, local, or regional efforts to publicly report on healthcare performance across the United States.³⁰

To better understand the public-reporting activities in a subset of these community-based groups, NQF analyzed the measure use of 16 alliances that receive funding from the Robert Wood Johnson Foundation through the Aligning Forces for Quality (AF4Q) program. This analysis showed that these alliances are using 171 NQF-endorsed measures in their reports to the public, and it provided insight to NQF as to the kinds of tools and capabilities communities are seeking as they evolve measurement efforts on the local level.

Supported by the Robert Wood Johnson Foundation, NQF has developed tools outside of the HHS contracts to support local, state, and regional leaders interested in using NQF-endorsed measures, particularly those measures also used in federal programs. For example, NQF's publicly available Quality Positioning System (QPS) enables users to search a database of NQF-endorsed measures and to build a portfolio or custom list of NQF-endorsed measures that they use or in which they are interested. A QPS user can then compare that portfolio against measures used in federal and other national programs, aligning measurement efforts where it makes sense to do so. A QPS user also can share its portfolio with others by self-publishing it within QPS on the NQF Web site. This feature and the ability to discern which NQF-endorsed measures are being used in federal programs can provide a rich information base to help communities, states, and the federal government synchronize their approaches to measuring and improving quality.

5. Identifying Measure Gaps and Developing Strategies for Filling Them

Under section 1890(b)(5)(iv) of the Social Security Act, the entity is required to describe gaps in endorsed quality and efficiency measures, including measures within priority areas identified by HHS under the agency's National Quality Strategy, and where quality and efficiency measures are unavailable or inadequate to identify or address such gaps. Under section 1890(b)(5)(v) of the Social Security Act, NQF is also required to describe areas in which evidence is insufficient to support endorsement of quality and efficiency measures in priority areas identified by the National Quality Strategy and where targeted research may address such gaps.

Performance measurement science has made important strides in the last decade, including addressing new settings and types of providers, becoming more responsive to the needs and preferences of varied stakeholders, evolving with new technology, and increasingly addressing hard-to-measure concepts such as care coordination and appropriateness. Despite these gains, measurement gaps persist, either because the measures have not yet been developed, or the measures exist but are not being used.

To identify measurement gaps, NQF conducted an extensive analysis in 2012 of its current measures portfolio against both the National Quality Strategy priority areas and high-impact conditions (both Medicare and child health) as required by statute (Social Security Act, section 1890(b)(5)(iv)), analyzed stakeholder feedback, and considered which NQF-endorsed measures were being used and by which sector. The gaps identified below, however, do need to be viewed in the context of rising concern about measurement overload and administrative burden. While more measures are needed to address high-priority issues, NQF continues to remove measures that no longer meet its criteria or where performance “tops out” to ensure measurement parsimony.

Synthesis of Measure Gaps

Captured in the 2012 NQF Measure Gap Analysis, this report revealed that discussions of measure gaps too often remain at a high conceptual level, and that more detailed information is needed to inform next steps, whether those steps entail measure development or addressing barriers to implementation of existing measures. In addition, while there may be non-NQF endorsed measures currently in use that address high-priority gap areas, a full assessment of their applicability and appropriateness was beyond the scope of this project. Such measures should be brought forth for NQF endorsement to assess their importance, scientific reliability and validity, usability, and feasibility before an assessment of value or recommendations for use can be made. The following are high-level syntheses of the measure gaps identified through the NQF analysis, presented through the lens of the three aims of the NQS.

Better Care

The lion's share of current NQF-endorsed measures related to better care focused on specific conditions. Addressing the gaps identified below would provide added input directly from patients about their care and could further focus the healthcare system on the needs and preferences of patients and families, including the most vulnerable patients.

Patient-reported outcomes (PROs)—To fully assess the quality and safety of healthcare, the gap analysis emphasized the importance of patient-reported outcomes—any report of the patient's health status that comes directly from the patient, without interpretation by a clinician or anyone else. Domains for measurement include symptoms and symptom burden, health-related quality of life including functional status, experience with care, and health-related behaviors. Especially important are PRO-based performance measures that can be aggregated accurately and reliably to the level of an accountable healthcare entity, and that span the full continuum of care.

Patient-centered care and shared decision-making—To spur the healthcare system to be more responsive to patients and families, measures are needed that assess whether patient and family treatment preferences are identified; whether their psychosocial, cultural, spiritual, or healthcare literacy needs are addressed; whether they are actively engaged in developing a care plan; and whether their expressed preferences and goals for care are met. Measures of decision quality are critical for assessing whether patients understand evidence-based treatment options and whether they are able to make decisions based on information provided by their healthcare practitioner.

Care coordination and care transitions—Important outcome measures are needed to assess whether patients, families, and caregivers believe that the overall care coordination process—including the quality of communication, care planning, care transitions, and team-based care—satisfactorily prepared them to manage their care and return to the best possible quality of life. The timeliness of access to high-quality palliative care or hospice services, including pain and symptom management, psychosocial support, and advance care planning also is identified as a gap area in need of further attention. Measure gaps related to effective medication management and patient adherence, and adverse drug events remain.

Care for vulnerable populations—A critical gap area to be filled includes the ability to measure whether high-quality care is available to patients most in need, particularly the vulnerable elderly, individuals with multiple chronic conditions and complex care needs, critically ill patients, patients receiving end-of-life care, children with special needs, residents in long-term care settings, the homeless, and people who are dually eligible for Medicare and Medicaid.

Healthy People/Healthy Communities

Recognizing that the health of the American public is mostly attributable to healthy life style behaviors, environment, or social status, the following gap areas push the field beyond the traditional boundaries of the healthcare delivery system and offer the potential for dramatic gains in health for the nation.

Health and well-being—Measures within and outside of the healthcare system are needed to assess health-related quality of life and to optimize the population's well-being. Measures that assess the burden of illness experienced by patients, families, and caregivers, as well as measures of productivity also are important. Community indices that measure key factors or social determinants known to significantly influence health or drive unnecessary utilization of healthcare services are needed to develop community programs that effectively and appropriately target resources and interventions to improve population health and reduce disparities.

Preventive care—Composite measures of the highest impact age- and sex-appropriate clinical preventive services, particularly for the cardiovascular disease priority area, continue to be important measure gaps to fill. Oral health was highlighted as an important area in need of measures, specifically for the prevention of dental caries, as were coordination of long-term support services and psychosocial, behavioral health, spiritual, and cultural services. An emerging area of focus for measurement is on the extent to which care is coordinated beyond the healthcare delivery system—particularly between healthcare, public health, and community support services—and how individual organizations are held collectively accountable.

Childhood measures—Measure gaps for child and adolescent health emphasized the attainment of developmental milestones, the quality of adolescent well-care visits, prevention of accidents and injuries, and prevention of risky behaviors. There also is a heightened need for measures of childhood obesity in addition to body mass index for more effective upstream management, given the risk for development of diabetes, cardiovascular disease, and other chronic conditions.

Accessible and Affordable Care

Affordability is often narrowly construed. The following identification of gaps broadens its definition so that affordability is viewed through a variety of lenses including the individual and society, for example, out-of-pocket costs to patients and families and costs to the healthcare system. Further, a commitment to ensuring access to affordable, high quality care for all necessitates judicious use of resources at the individual level.

Access to care—In addition to measures that assess insurance coverage, the analysis revealed that measure gaps indicative of access to needed care are important to address. Important considerations include the ability to obtain medications, mental health, oral health, and specialty services in a timely fashion. Measures also are needed to assess disparities in access and affordability, particularly with regard to socioeconomic status, race, and ethnicity, and for vulnerable populations.

Healthcare affordability—Many stakeholders emphasize the need for affordability indices that reflect the burden of healthcare costs on consumers and that include direct costs (e.g., out-of-pocket expenses, personal healthcare expenditures per capita) as well as indirect opportunity costs (e.g., productivity, work and school absenteeism, and the “cost of neglect” of medical and dental care). Efficiency measures are needed to benchmark providers on cost and quality as well as to quantify the impact of inefficiencies across care settings to further target quality improvement efforts. Purchasers and consumers continue to emphasize the importance of understanding pricing and improved transparency of data through standardized measurement and reporting.

Waste and overuse—Measures that assess the extent to which the healthcare system promotes the provision of medical, surgical, and diagnostic services that offer little if any value—and that may be harmful to patients—are critical to closing gaps in variation. Specific areas frequently cited as important for measurement include appropriate, patient-centered and patient-directed end-of-life care; unnecessary emergency department visits and hospital admissions and readmissions (particularly for ambulatory-sensitive conditions); inappropriate medication use and polypharmacy; and duplication of or inappropriate services and testing, particularly imaging.

Availability of NQF-endorsed Measures

Although the NQF portfolio increasingly maps to the NQS, its extent varies across each of the six NQS priorities. For example, 40 percent of NQF measures that map to the NQS at the goal level address patient safety, including a wide range of measures related to healthcare-acquired conditions and hospital readmissions. Yet only 7 percent of measures that map at the goal level address patient and family engagement, with very few measures to address important areas of shared decision making, patient navigation, and patient self-management. Likewise, measures to address healthy lifestyle behaviors and community interventions to prevent cardiovascular disease upstream also warrant increased attention. Specific measures of cost remain a high-priority gap area, particularly for purchasers of healthcare.

NQF's portfolio includes more than 400 condition-specific measures, more than 250 of which address the high-impact Medicare conditions. Yet only 53 of the measures address the specific high-impact child health conditions, and 12 of the high-impact child health conditions do not have any specific endorsed measures. While the lack of measures for certain conditions may be of interest or concern, future measure development should be prioritized to focus on cross-cutting measures that apply to patients regardless of their disease process.

NQF Measure Portfolio in Use

The federal government remains the predominant user of NQF-endorsed measures, but a growing number of measures are in use across other public-sector programs—including state and local programs—as well as in the private sector. More promising is the emerging overlap in measure use across these sectors. Further alignment—or use of the same measures—offers the potential to significantly reduce measurement burden and to simultaneously accelerate improvement by sending consistent signals about what is important for providers to focus care improvement resources against.

Overall, 64 measures in the NQF portfolio that address specific NQS goals are in concurrent use in federal programs and two or more private programs. While the majority of these are safety-related measures, a small number address aspects of overuse, patient experience, and preventive screenings. A nearly equal number of measures that address specific NQS goals are not in use in any of the programs analyzed—a missed opportunity, particularly for goals related to function and quality of life, hospice and palliative care, mental health, and preventive services for children. Similarly, the analysis revealed that 57 measures in the NQF portfolio that address high-impact conditions are in concurrent use in federal programs and two or more private programs, the majority of which reflect the high-impact Medicare conditions. However, 47 measures that address high-impact Medicare or child health conditions had no identified use in any of the sectors analyzed. Consideration should be given to the potential barriers that prevent these measures from being implemented in the field.

The Path Forward

As the field—the public and private stakeholders committed to building a solid foundation for quality improvement—strives to continually advance the use of standardized performance measurement, there is a strong desire to accelerate efforts to fill, rather than just identify, key measurement gaps. This will require making better use of the measures already available for key priority areas and investing wisely in measure development and endorsement activities to fill the most critical gap areas.

6. Looking Forward

NQF has evolved in the dozen years it has been in existence and since it endorsed its first performance measures a decade ago. While its focus on improving quality, enhancing safety, and reducing costs by endorsing performance measures has remained a constant, its role has expanded to include a significant emphasis on getting the various stakeholder groups to align with respect to their use of performance measures and related improvement efforts. Experience has made it clear that sector-by-sector approaches to enhancing healthcare performance are ineffective in our decentralized and complex healthcare system, and they waste precious healthcare resources and may even do harm.

Looking ahead, NQF will work together with HHS and the broader quality movement to:

• Deepen the alignment between the public and private sectors and across stakeholder groups to accelerate progress and reduce burden: This relates to measure endorsement and the work of NQF-convened partnerships and is a core, enduring value of the organization;
• Focus more on “end user” needs and engagement: NQF will enlarge its current collaborative efforts to better incorporate the perspectives and values of those at the local level and those on the sharp end of healthcare—who ultimately are integrating the needs of the delivery system with those who receive and pay for care. Starting with the preferences of the end user in mind and systematically collecting user feedback about the efficacy of measures are ways to engage communities, providers, and other users in the collective goal of improving healthcare value.
• Take a more proactive approach to coordinate the measures pipeline and remake measure review and endorsement so it is more nimble: NQF will not only identify measure gaps but engage developers in filling them so that their efforts are streamlined and avoid duplication. Simultaneously, NQF plans to set up standing committees so that measures can more readily be reviewed.
• Review and endorse “next generation” quality measures that put the patient first: A key priority is endorsing next-generation measures that are more meaningful to patients and families and that help track patient outcomes across healthcare settings. NQF is committed to moving our nation's healthcare system to be ever more responsive to patient preferences and values and believes that richer information can play a crucial role;
• Increase the focus on measures that can enhance value: Affordability and its relationship to quality will become a focal point and better integrated into NQF's future work, starting with defining the many aspects of affordability and prioritizing near and longer term areas of focus going forward. Given the embryonic stage of affordability measures overall, there is much upfront conceptual work to be done that will rely on getting broad-based and varied input in order to gain a deeper appreciation for how to further measurement in the areas of costs, appropriateness, and resource use and how to pair such measures with quality metrics in order to assess value.

NQF is embarking on an exciting agenda that emphasizes enhanced alignment and collaboration so as to better integrate end user needs—all with an eye on evolving our measure portfolio so that it drives the healthcare system toward both delivering higher value healthcare and incorporating the needs and preferences of patients, payers, and purchasers. The goals are clear, and the collective work of the 800 plus individuals who collaborate with NQF are focused on efforts to benefit the U.S. healthcare system and the patients it serves.

Appendix A: 2012 Accomplishments

Appendix B: NQF Board and Management Team

Board of Directors

William L. Roper, MD, MPH (Chair), Dean, School of Medicine, Vice Chancellor for Medical Affairs and Chief Executive Officer, UNC Health Care System, University of North Carolina at Chapel Hill

Helen Darling, MA (Vice Chair), President, National Business Group on Health

Gerald M. Shea (Treasurer and Interim CEO), Assistant to the President for External Affairs, AFL-CIO

Lawrence M. Becker, Director, HR Strategic Partnerships, Xerox Corporation

JudyAnn Bigby, MD, Secretary, Executive Office of Health & Human Services, Commonwealth of Massachusetts

Jack Cochran, MD, FACS, Executive Director, The Permanente Federation

Maureen Corry, Executive Director, Childbirth Connection

Leonardo Cuello, Staff Attorney, National Health Law Program

Joyce Dubow, Senior Health Care Reform Director, AARP Office of the Executive Vice-President for Policy and Strategy

Robert Galvin, MD, MBA, Chief Executive Officer, Equity Healthcare, The Blackstone Group

Ardis Dee Hoven, MD, Chair, Board of Trustees, American Medical Association

Charles N. Kahn III, MPH, President, Federation of American Hospitals

Donald Kemper, Chairman and CEO, Healthwise, Inc.

William Kramer, Executive Director for National Health Policy, Pacific Business Group on Health

Harold D. Miller, President and CEO, Network for Regional Healthcare Improvement

Elizabeth Mitchell, CEO, Maine Health Management Coalition

Dolores L. Mitchell, Executive Director, Commonwealth of Massachusetts Group Insurance Commission

Mary Naylor, Ph.D., RN, FAAN, Director, New Courtland Center for Transitions & Health and Marian S. Ware Professor in Gerontology, University of Pennsylvania School of Nursing

Debra L. Ness, President, National Partnership for Women & Families

Samuel R. Nussbaum, MD, Executive Vice President and Chief Medical Officer, WellPoint, Inc.

J. Marc Overhage, MD, Ph.D., Chief Medical Informatics Officer, Siemens Medical Solutions, Inc.

Bernard M. Rosof, MD, Chair, Board of Directors, Huntington Hospital, Chair, Physician Consortium for Performance Improvement (PCPI)

John C. Rother, JD, President and CEO, National Coalition on Health Care

Bruce Siegel, MD, MPH, President and Chief Executive Officer, National Association of Public Hospitals and Health Systems (NAPH)

Joseph R. Swedish, FACHE, President and CEO, Trinity Health

John Tooker, MD, MBA, MACP, Associate Executive Vice President, American College of Physicians

Richard J. Umbdenstock, FACHE, President and CEO, American Hospital Association

CMS

Patrick Conway, MD, Chief Medical Officer, Centers for Medicare & Medicaid Services

AHRQ

Carolyn M. Clancy, MD, Director, Agency for Healthcare Research and Quality

Designee: Nancy Wilson, MD, MPH, Senior Advisor to the Director

HRSA

Mary Wakefield, Ph.D., RN, Administrator, Health Resources and Services Administration

Designee: Terry Adirim, MD, Director, Office of Special Health Affairs

CDC

Thomas R. Frieden, MD, MPH, Director, Centers for Disease Control and Prevention

Designee: Peter A. Briss, MD, MPH, Captain, U.S. Public Health Service, Medical Director

EX OFFICIO (NON-VOTING):

Ann Monroe, (Chair, Consensus Standards Approval Committee), President, Health Foundation for Western and Central New York

Paul C. Tang, MD, MS, (Chair, Health Information Technology Advisory Committee) Vice President and Chief Medical Information Officer Palo Alto Medical Foundation

Management Team

Gerald Shea, Interim Chief Executive Officer

Karen Adams, Vice President, National Priorities

Heidi Bossley, Vice President, Performance Measures

Helen Burstin, Senior Vice President, Performance Measures

Ann Greiner, Vice President, Government Relations

Ann Hammersmith, General Counsel

Lisa Hines, Vice President, Member Relations

Rosemary Kennedy, Vice President, Health Information Technology

Nicole Silverman, Vice President, Program Operations

Lindsey Spindle, Senior Vice President, Communications and External Affairs

Diane Stollenwerk, Vice President, Stakeholder Collaboration

Jeffrey Tomitz, Chief Financial Officer, Accounting & Finance

Thomas Valuck, Senior Vice President, Strategic Partnerships

Kyle Vickers, Chief Information Office

Appendix C: MAP “Working” Measure Selection Criteria

1. Measures Within the Program Measure Set Are NQF-endorsed or Meet the Requirements for Expedited Review

Measures within the program measure set are NQF-endorsed, indicating that they have met the following criteria: important to measure and report, scientifically acceptable measure properties, usable, and feasible. Measures within the program measure set that are not NQF-endorsed but meet requirements for expedited review, including measures in widespread use and/or tested, may be recommended by MAP, contingent on subsequent endorsement. These measures will be submitted for expedited review.

Response option: Strongly Agree/Agree/Disagree/Strongly Disagree

Measures within the program measure set are NQF-endorsed or meet requirements for expedited review (including measures in widespread use and/or tested)

Additional Implementation Consideration: Individual endorsed measures may require additional discussion and may be excluded from the program measure set if there is evidence that implementing the measure would result in undesirable unintended consequences.

2. Program Measure Set Adequately Addresses Each of the National Quality Strategy (NQS) priorities

Demonstrated by measures addressing each of the National Quality Strategy (NQS) priorities:

Subcriterion 2.1 Safer care

Subcriterion 2.2 Effective care coordination

Subcriterion 2.3 Preventing and treating leading causes of mortality and morbidity

Subcriterion 2.4 Person- and family-centered care

Subcriterion 2.5 Supporting better health in communities

Subcriterion 2.6 Making care more affordable

Response option for each subcriterion: Strongly Agree/Agree/Disagree/Strongly Disagree:

NQS priority is adequately addressed in the program measure set

3. Program Measure Set Adequately Addresses High-impact Conditions Relevant to the Program's Intended Population(s) (e.g., Children, Adult non-Medicare, Older Adults, Dual Eligible Beneficiaries)

Demonstrated by the program measure set addressing Medicare High-Impact Conditions; Child Health Conditions and risks; or conditions of high prevalence, high disease burden, and high cost relevant to the program's intended population(s). (Refer to tables 1 and 2 for Medicare High-Impact Conditions and Child Health Conditions determined by the NQF Measure Prioritization Advisory Committee.)

Response option: Strongly Agree/Agree/Disagree/Strongly Disagree:

Program measure set adequately addresses high-impact conditions relevant to the program.

4. Program Measure Set Promotes Alignment With Specific Program Attributes, as Well as Alignment Across Programs

Demonstrated by a program measure set that is applicable to the intended care setting(s), level(s) of analysis, and population(s) relevant to the program.

Response option for each subcriterion:

Strongly Agree/Agree/Disagree/Strongly Disagree

Subcriterion 4.1 Program measure set is applicable to the program's intended care setting(s)

Subcriterion 4.2 Program measure set is applicable to the program's intended level(s) of analysis

Subcriterion 4.3 Program measure set is applicable to the program's population(s)

5. Program Measure Set Includes an Appropriate Mix of Measure Types

Demonstrated by a program measure set that includes an appropriate mix of process, outcome, experience of care, cost/resource use/appropriateness, and structural measures necessary for the specific program attributes.

Response option for each subcriterion: Strongly Agree/Agree/Disagree/Strongly Disagree

Subcriterion 5.1 Outcome measures are adequately represented in the program measure set

Subcriterion 5.2 Process measures are adequately represented in the program measure set

Subcriterion 5.3 Experience of care measures are adequately represented in the program measure set (e.g. patient, family, caregiver)

Subcriterion 5.4 Cost/resource use/appropriateness measures are adequately represented in the program measure set

Subcriterion 5.5 Structural measures and measures of access are represented in the program measure set when appropriate

6. Program Measure Set Enables Measurement Across the Person-Centered Episode of Care

Demonstrated by assessment of the person's trajectory across providers, settings, and time.

Response option for each subcriterion: Strongly Agree/Agree/Disagree/Strongly Disagree

Subcriterion 6.1 Measures within the program measure set are applicable across relevant providers

Subcriterion 6.2 Measures within the program measure set are applicable across relevant settings

Subcriterion 6.3 Program measure set adequately measures patient care across time

7. Program Measure Set Includes Considerations for Healthcare Disparities

Demonstrated by a program measure set that promotes equitable access and treatment by considering healthcare disparities. Factors include addressing race, ethnicity, socioeconomic status, language, gender, age disparities, or geographical considerations (e.g., urban vs. rural). Program measure set also can address populations at risk for healthcare disparities (e.g., people with behavioral/mental illness).

Response option for each subcriterion: Strongly Agree/Agree/Disagree/Strongly Disagree

Subcriterion 7.1 Program measure set includes measures that directly assess healthcare disparities (e.g., interpreter services)

Subcriterion 7.2 Program measure set includes measures that are sensitive to disparities measurement (e.g., beta blocker treatment after a heart attack)

8. Program Measure Set Promotes Parsimony

Demonstrated by a program measure set that supports efficient (i.e., minimum number of measures and the least effort) use of resources for data collection and reporting and supports multiple programs and measurement applications. The program measure set should balance the degree of effort associated with measurement and its opportunity to improve quality.

Response option for each subcriterion: Strongly Agree/Agree/Disagree/Strongly Disagree

Subcriterion 8.1 Program measure set demonstrates efficiency (i.e., minimum number of measures and the least burdensome)

Subcriterion 8.2 Program measure set can be used across multiple programs or applications (e.g., Meaningful Use, Physician Quality Reporting System [PQRS])

Appendix D: 2012 NQF Expert Participant Leaders (organized by committee)

Behavioral Health Steering Committee

Peter Briss, Co-Chair, National Center for Chronic Disease Prevention and Health Promotion

Harold Pincus, Co-Chair, Columbia University

Cancer Steering Committee

Stephen Edge, Co-Chair, Roswell Park Cancer Institute

Stephen Lutz, Chair, Blanchard Valley Regional Cancer Center

Cardiovascular Endorsement Maintenance 2010 Steering Committee

Mary George, Vice Chair, Centers for Disease Control and Prevention

Raymond Gibbons, Chair, Mayo Clinic

Care Coordination Steering Committee

Donald Casey, Co-Chair, Atlantic Health

Gerri Lamb, Co-Chair, Arizona State University

Common Formats Expert Panel

David Classen, Co-Chair, University of Utah School of Medicine

Henry Johnson, Co-Chair, ACS-MIDAS+

Council Leadership

Tanya Alteras, Chair, National Partnership for Women & Families

Maureen Corry, Vice Chair, Childbirth Connection

Deborah Fritz, Vice Chair, GlaxoSmithKline

Seiji Hayashi, Chair, Health Resources and Services Administration

David Hopkins, Chair, Pacific Business Group on Health

Thomas James, Chair, Humana Inc.

Carol Mullin, Chair, Virtua Health

Michael Phelan, Vice Chair, Cleveland Clinic

Louise Probst, Vice Chair, St. Louis Area Business Health Coalition

William Rich, Chair, Northern Virginia Ophthalomology Associates

Richard Salmon, Vice Chair, CIGNA HealthCare

David Shahian, Vice Chair, Massachusetts General Hospital

Kathleen Shoemaker, Chair, Lilly USA, LLC

Hussein Tahan, Vice Chair, New York Presbyterian Healthcare System

Marcia Wilson, Chair, Center for Health Care Quality

CSAC: Consensus Standards Approval Committee

Ann Monroe, Chair, Vice Chair, Health Foundation for Central & Western New York

Frank Opelka, Vice Chair, American College of Surgeons

GI & GU Pilot Project Steering Committee

Andrew Baskin, Co-Chair, Aetna

Christopher Saigal, Co-Chair, UCLA Medical Center

Health Information Technology Advisory Committee

J. Marc Overhage, Vice Chair, Siemens Medical Solutions USA, Inc.

Paul Tang, Chair, Palo Alto Medical Foundation

Healthcare Disparities & Cultural Competency Steering Committee

Dennis Andrulis, Co-Chair, Texas Health Institute

Denice Cora-Bramble, Co-Chair, Children's National Medical Center

HITAC Change Control Board

Floyd Eisenberg, Chair, NQF

HITAC Oversight and Testing Workgroup

Michael Lieberman, Chair, Oregon Health and Sciences University

HITAC Quality Data Model Subcommittee

David Bates, Chair, Brigham and Women's Hospital

Caterina Lasome, Co-Chair, iON Informatics, LLC

Infectious Disease Steering Committee

Steven Brotman, Co-Chair, The Advanced Medical Technology Association (AdvaMed)

Edward Septimus, Co-Chair, HCA

Leadership Network

William Corley, Chair, Community Health Network

MAP Cardiovascular and Diabetes Care Task Force

Christine Cassel, Chair, American Board of Internal Medicine

MAP Safety and Care Coordination Task Force

Frank Opelka, Chair, American College of Surgeons

MAP Strategy Task Force 2

Charles Kahn, Co-Chair, Federation of American Hospitals

Gerald Shea, Co-Chair, AFL-CIO

Measure Applications Partnership Clinician Workgroup

Mark McClellan, Chair, The Brookings Institute

Measure Applications Partnership Coordinating Committee

George Isham, Co-Chair, HealthPartners

Elizabeth McGlynn, Co-Chair, Kaiser Permanente Center for Effectiveness & Safety Research

Measure Applications Partnership Dual Eligibles Workgroup

Alice Lind, Chair, Center for Health Care Strategies, Inc

Measure Applications Partnership Hospital Workgroup

Frank Opelka, Chair, American College of Surgeons

Measure Applications Partnership PAC-LTC Workgroup

Carol Raphael, Chair, Visiting Nurse Service of New York

Multiple Chronic Conditions Measurement Framework Steering Committee

Caroline Blaum, Co-Chair, DVAMC GRECC Institute of Gerontology

Barbara McCann, Co-Chair, Interim HealthCare Inc.

National Priorities Partnership

Helen Darling, Co-Chair, National Business Group on Health

Bernard Rosof, Co-Chair, American Medical Association-Physician Consortium for Performance Improvement

Neurology Steering Committee

David Knowlton, Co-Chair, New Jersey Health Care Quality Institute

David Tirschwell, Co-Chair, University of Washington, Department of Neurology

NPP Maternity Action Team

Maureen Corry, Co-Chair, Childbirth Connection

Bernard Rosof, Co-Chair, American Medical Association-Physician Consortium for Performance Improvement

NPP Readmissions Action Team

Helen Darling, Co-Chair, National Business Group on Health

Susan Frampton, Co-Chair, Planetree

Oral Health Expert Panel

Paul Glassman, Co-Chair, University of the Pacific School of Dentistry

David Krol, Co-Chair, The Robert Wood Johnson Foundation

Palliative Care and End of Life Care Steering Committee

June Lunney, Co-Chair, Hospice and Palliative Nurses Association

Sean Morrison, Co-Chair, Mount Sinai School of Medicine—Dept. of Geriatrics & Palliative Medicine

Patient Safety State Based Reporting Work Group

Michael Doering, Co-Chair, Pennsylvania Patient Safety Authority

Diane Rydrych, Co-Chair, Minnesota Department of Health

Iona Thraen, Co-Chair, Utah Department of Health

Patient Safety-Measures Complications Steering Committee

Pamela Cipriano, Co-Chair, University of Virginia Health System

William Conway, Co-Chair, Henry Ford Health System

Perinatal and Reproductive Health Steering Committee

Laura Riley, Co-Chair, Massachusetts General Hospital

Carol Sakala, Co-Chair, Childbirth Connection

Population Health Steering Committee

Paul Jarris, Co-Chair, Association of State and Territorial Health Officers

Kurt Stange, Co-Chair, Case Western Reserve University

Pulmonary Steering Committee

Stephen Grossbart, Co-Chair, Catholic Health Partners

Kevin Weiss, Co-Chair, American Board of Medical Specialties

Readmissions Expedited Review Steering Committee

Sherrie Kaplan, Co-Chair, UC Irvine School of Medicine

Eliot Lazar, Co-Chair, New York Presbyterian Healthcare System

Regionalized Emergency Medical Care Services Steering Committee

Arthur Kellermann, Co-Chair, The RAND Corporation

Andrew Roszak, Co-Chair, HHS\HRSA

Resource Use Project Cancer TAP

David Penson, Chair, Vanderbilt University Medical Center

Resource Use Project Cardio/Diab TAP

Jeptha Curtis, Co-Chair, Yale University School of Medicine

James Rosenzweig, Co-Chair, Boston Medical Center and Boston University School of Medicine

Resource Use Project: Bone/Joint TAP

James Weinstein, Chair, Dartmouth-Hitchcock Medical Center

Resource Use Project: Pulmonary TAP

Kurtis Elward, Co-Chair, Family Medicine of Albermarle

Janet Maurer, Co-Chair, American College of Chest Physicians

Appendix E: 2012 NQF Expert Participants (organized by affiliation)

Barbara Kelly—A.F. Williams Family Medicine Center

Joyce Dubow—AARP

Naomi Karp—AARP

Susan Reinhard—AARP

Judith Cahill—Academy of Managed Care Pharmacy

Marissa Schlaifer—Academy of Managed Care Pharmacy

Henry Johnson—ACS-MIDAS+

Madhavi Vemireddy—ActiveHealth Management

Henry Claypool—Administration for Community Living, HHS

Joanne Armstrong—Aetna

Andrew Baskin—Aetna

Thomas Howe—Aetna

Randall Krakauer—Aetna

Patricia McDermott—Aetna

Gerald Shea—AFL-CIO

Marie Kokol—Agency for Health Care Administration

Carolyn Clancy—Agency for Healthcare Research and Quality

Erin Grace—Agency for Healthcare Research and Quality

Darryl Gray—Agency for Healthcare Research and Quality

Ernest Moy—Agency for Healthcare Research and Quality

William Munier—Agency for Healthcare Research and Quality

Mary Nix—Agency for Healthcare Research and Quality

Mamatha Pancholi—Agency for Healthcare Research and Quality

D.E.B. Potter—Agency for Healthcare Research and Quality

Judith Sangl—Agency for Healthcare Research and Quality

Nancy Wilson—Agency for Healthcare Research and Quality

MaryAnne Lindeblad—Aging and Disability Services Administration

Sam Fazio—Alzheimer's Association

Beth Kallmyer—Alzheimer's Association

Julie Lewis—Amedisys

Bruce Bagley—American Academy of Family Physicians

Dennis Saver—American Academy of Family Physicians

Dale Lupu—American Academy of Hospice and Palliative Medicine

Jack Scariano—American Academy of Neurology

Mary Jo Goolsby—American Academy of Nurse Practitioners

Douglas Burton—American Academy of Orthopaedic Surgeons

John Ratliff—American Association of Neurological Surgeons

Christine Zambricki—American Association of Nurse Anesthetists

Margaret Nygren—American Association on Intellectual and Developmental Disabilities

Christine Cassel—American Board of Internal Medicine

Lorna Lynn—American Board of Internal Medicine

Denece Kesler—American Board of Medical Specialties

Kevin Weiss—American Board of Medical Specialties

Larry Gilstrap—American Board of Obstetrics and Gynecology

Mary Maryland—American Cancer Society Illinois Division

Janet Maurer—American College of Chest Physicians

Lisa Moores—American College of Chest Physicians

Lorrie Kaplan—American College of Nurse-Midwives

Sean Currigan—American College of Obstetricians and Gynecologists

Gerald Joseph—American College of Obstetricians and Gynecologists

Sandra Fryhofer—American College of Physicians

Amir Qaseem—American College of Physicians

Don Detmer—American College of Surgeons

Bruce Hall—American College of Surgeons

Frank Opelka—American College of Surgeons

Sally Tyler—American Federation of State, County and Municipal Employees

Jennie Hansen—American Geriatrics Society

David Gifford—American Health Care Association

Ruta Kadonoff—American Health Care Association

Naomi Naierman—American Hospice Foundation

Nancy Foster—American Hospital Association

Richard Umbdenstock—American Hospital Association

Kalpana Ramiah—American Institutes for Research

Norman Edelman—American Lung Association

Kendra Hanley—American Medical Association

Delane Heldt—American Medical Association-Physician Consortium for Performance Improvement

Bernard Rosof—American Medical Association-Physician Consortium for Performance Improvement

James Lett—American Medical Directors Association

Sam Lin—American Medical Group Association

Maureen Dailey—American Nurses Association

Marla Weston—American Nurses Association

Patricia Conway-Morana—American Organization of Nurse Executives

Dianne Jewell—American Physical Therapy Association

Arden Morris—American Society of Colon and Rectal Surgeons

Shekhar Mehta—American Society of Health-System Pharmacists

Janet Brown—American Speech-Language-Hearing Association

Aparna Higgins—America's Health Insurance Plans

Andrea Gelzer—AmeriHealth Mercy Family of Companies

Richard Dutton—Anesthesia Quality Institute

Jay Schukman—Anthem Blue Cross and Blue Shield

Michael Helgeson—Apple Tree Dental

Gerri Lamb—Arizona State University

Craig Gilliam—Arkansas Children's Hospital

Catherine Tapp—Arkansas Department of Health and Human Services

Ann Hendrich—Ascension Health

Sarah Hill—Ascension Health

Joanne Conroy—Association of American Medical Colleges

Marilyn Bowman-Hayes—Association of periOperative Registered Nurses

Paul Jarris—Association of State and Territorial Health Officers

Shawn Polk—Association of State and Territorial Health Officials

Donald Casey—Atlantic Health

Michael Cantine—Atlantic Health

Roger Kurlan—Atlantic Health

Rhonda Anderson—Banner Health System

Ann de Velasco—Baptist Health South Florida

Thomas Giordano—Baylor College of Medicine

Jochen Profit—Baylor College of Medicine

Carl Couch—Baylor Health Care System

Jean De Leon—Baylor Health Care System

Robert Fine—Baylor Health Care System

Robert Watson—Baylor Health Care System

David Hackney—Beth Israel Deaconess Medical Center

Nancy Ridley—Betsy Lehman Center for Patient Safety and Medical Error Reduction

Patrick Murray—Better Health Greater Cleveland

Debra Bakerjian—Betty Irene Moore School of Nursing

Tiffany Osborn—BJC HealthCare

Stephen Lutz—Blanchard Valley Regional Cancer Center

Jane Franke—Blue Cross Blue Shield of Massachusetts

Greg Pawlson—BlueCross BlueShield Association

Carol Wilhoit—BlueCross BlueShield of Illinois

Kristine Anderson—BoozAllenHamilton

George Philippides—Boston Medical Center

James Rosenzweig—Boston Medical Center

Jeffrey Samet—Boston University School of Medicine

Lewis Kazis—Boston University School of Public Health

David Bates—Brigham and Women's Hospital

Daniel Forman—Brigham and Women's Hospital

Bruce Koplan—Brigham and Women's Hospital

Jeffrey Greenberg—Brigham and Women's Physicians' Organization

Richard Zane—Brigham Women's Hospital

Barbara Caress—Building Services 32BJ Health Fund

Lisa Shea—Butler Hospital

Carolyn Pare—Buyers Health Care Action Group

Neal Kohatsu—California Department of Health Care Services

Loriann DeMartini—California Department of Public Health

Kathleen O'Malley—California HealthCare Foundation

Ellen Wu—California Pan-Ethnic Health Network

Evelyn Calvillo—California State University

Janet Young—Carilion Health Systems

Jennifer Brandenburg—Carle Foundation Hospital

Suzanne Snyder—Carolinas Rehabilitation

Kurt Stange—Case Western Reserve University

Suzanne Delbanco—Catalyst for Payment Reform

Gail Amundson—Caterpillar Inc.

Stephen Grossbart—Catholic Health Partners

Zab Mosenifar—Cedars Sinai Medical Center

Kimberly Gregory—Cedars-Sinai Medical Center

Michael Langberg—Cedars-Sinai Medical Center

Rekha Murthy—Cedars-Sinai Medical Center

David Palestrant—Cedars-Sinai Medical Center

Marcia Wilson—Center for Health Care Quality, Department of Health Policy, George Washington University

Alice Lind—Center for Health Care Strategies, Inc

Elliot Sloane—Center for Healthcare Information Research and Policy

Arthur Levin—Center for Medical Consumers

Alfred Chiplin Jr.—Center for Medicare Advocacy, Inc.

Patricia Nemore—Center for Medicare Advocacy, Inc.

Terrence Batliner—Center for Native Oral Health Research

Diane Meier—Center to Advance Palliative Care

Peter Briss—Centers for Disease Control and Prevention

William Callaghan—Centers for Disease Control and Prevention

Mary George—Centers for Disease Control and Prevention

Catherine Gordon—Centers for Disease Control and Prevention

Gail Janes—Centers for Disease Control and Prevention

Chesley Richards—Centers for Disease Control and Prevention

Patrick Conway—Centers for Medicare & Medicaid Services

Maria Durham—Centers for Medicare & Medicaid Services

Kate Goodrich—Centers for Medicare & Medicaid Services

Shaheen Halim—Centers for Medicare & Medicaid Services

Shari Ling—Centers for Medicare & Medicaid Services

Cheryl Powell—Centers for Medicare & Medicaid Services

Michael Rapp—Centers for Medicare & Medicaid Services

Ashley Ridlon—Centers for Medicare & Medicaid Services

Marsha Smith—Centers for Medicare & Medicaid Services

Erin Smith—Centers for Medicare & Medicaid Services

Judith Tobin—Centers for Medicare & Medicaid Services

Alisa Ray—Certification Commission for Healthcare Information Technology

Parinda Khatri—Cherokee Health Systems

Maureen Corry—Childbirth Connection

Carol Sakala—Childbirth Connection

Ellen Schwalenstocker—Children's Hospital Association

Richard Antonelli—Children's Hospital Boston

Jenifer Lightdale—Children's Hospital Boston

Mark Schuster—Children's Hospital Boston

Trude Haecker—Children's Hospital of Philadelphia

David Einzig—Children's Hospitals and Clinics of Minnesota

Carol Kemper—Children's Mercy Hospital

Denice Cora-Bramble—Children's National Medical Center

David Stockwell—Children's National Medical Center

Joseph Wright—Children's National Medical Center

William Weintraub—Christiana Care Health System

Colette Edwards—CIGNA HealthCare

Mary Kay O'Neill—CIGNA HealthCare

Richard Salmon—CIGNA HealthCare

Uma Kotagal—Cincinnati Children's Hospital Medical Center

Thomas Loyacono—City of Baton Rouge and Parish of East Baton Rouge

Joseph Alvarnas—City of Hope

Jo Ann Brooks—Clarian Health

Jocelyn Bautista—Cleveland Clinic

Sung Hee Leslie Cho—Cleveland Clinic

Irene Katzan—Cleveland Clinic

David Lang—Cleveland Clinic

Thomas Marwick—Cleveland Clinic

Michael Phelan—Cleveland Clinic

Shannon Phillips—Cleveland Clinic

Allan Siperstein—Cleveland Clinic

Sharon Sutherland—Cleveland Clinic

Timothy Gilligan—Cleveland Clinic

Stanley Pestotnik—Cognovant, Inc.

Chris Tonozzi—Colorado Associated Community Health Information Enterprise

Kim Johnson—Colorado Department of Public Health and Environment

Wendy Tenzyk—Colorado Public Employees' Retirement Association

Arthur Cooper—Columbia University

Jacqueline Merrill—Columbia University

Bobbie Berkowitz—Columbia University School of Nursing

Lawrence Gottlieb—Commonwealth Care Alliance

Roger Snow—Commonwealth of Massachusetts

Dolores Mitchell—Commonwealth of Massachusetts —Group Insurance Commission

William Corley—Community Health Network

Andrea Benin—Connecticut Children's Medical Center

Cheryl Theriault—Connecticut Department of Health

Mary Alice Lee—Connecticut Voices for Children

E. Clarke Ross—Consortium for Citizens with Disabilities

Lawrence Sadwin—Consultant

Adam Thompson—Consultant

Richard Hanke—Consumer Representative

Robert Ellis—Consumers' Checkbook

Robert Krughoff—Consumers' Checkbook

Steven Findlay—Consumers Union

Lisa McGiffert—Consumers Union

Doris Peter—Consumers Union

Andrea Russo—Cooper University Hospital

Russell Acevedo—Crouse Hospital

Dolores Kelleher—D Kelleher Consulting

Richard Goldstein—Dana-Farber Cancer Institute

Saul Weingart—Dana-Farber Cancer Institute

John Wasson—Dartmouth-Hitchcock Medical Center

James Weinstein—Dartmouth-Hitchcock Medical Center

Linda Wilkinson—Dartmouth-Hitchcock Medical Center

Erik Pupo—Deloitte Consulting, LLP

Richard Albert—Denver Health Medical Center

Edward Havranek—Denver Health Medical Center

Philip Mehler—Denver Health Medical Center

Feseha Woldu—Department of Health and Human Services

Mary Sieggreen—Detroit Medical Center

Margaret Campbell—Detroit Receiving Hospital

Sharon Baskerville—District of Columbia Primary Care Association

Steve Morgenstern—Dow Chemical Company

Gwendolen Buhr—Duke University Health System

Sean O'Brien—Duke University Health System

John Clarke—ECRI Institute

Kathleen Shoemaker—Eli Lilly and Company

Nicole Tapay—Eli Lilly and Company

AnnMarie Papa—Emergency Nurses Association

Kathleen Szumanski—Emergency Nurses Association

Ricardo Martinez—Emory University School of Medicine

Amit Popat—Epic Systems Corp

Stanley Davis—Fairview Health Services

Brent Asplin—Fairview Medical Group

Kathleen Kelly—Family Caregiver Alliance

Kurtis Elward—Family Medicine of Albermarle

Allen McCullough—Fayette County Public Safety

Charles Kahn—Federation of American Hospitals

Nick Nudell—FirstWatch Solutions, Inc.

Joseph Ouslander—Florida Atlantic University

Laurie Burke—Food and Drug Administration

Jay Crowley—Food and Drug Administration

Behnaz Minaei—Food and Drug Administration

Terrie Reed—Food and Drug Administration

Terry Rogers—Foundation for Health Care Quality

Dwight Kloth—Fox Chase Cancer Center

Barbara Levy—Franciscan Health System

Dana Alexander—GE Healthcare

Brandon Savage—GE Healthcare

James Walker—Geisinger Health System

Andrew Guccione—George Mason University

Mayri Leslie—George Washington University

Robert Graham—George Washington University—School of Public Health

Michael Stoto—Georgetown University

Leslee Pool—Georgia Department of Health and Human Resources+D306

Rohit Borker—GlaxoSmithKline

Deborah Fritz—GlaxoSmithKline

Brenda Parker—GlaxoSmithKline

Richard Stanford—GlaxoSmithKline

John Derr—Golden Living, LLC

Connie Steed—Greenville Hospital System

Jason Colquitt—Greenway Medical Technologies

Anne Cohen—Harbage Consulting

John Gore—Harborview Medical Center

Ronald Maier—Harborview Medical Center

Paula Minton Foltz—Harborview Medical Center

David Spach—Harborview Medical Center

David Tirschwell—Harborview Medical Center

Jeffrey Greenwald—Harvard Medical School

Elsbeth Kalenderian—Harvard School of Dental Medicine

Ashish Jha—Harvard School of Public Health

Christine Klotz—Health Foundation for Central & Western New York

Ann Monroe—Health Foundation for Central & Western New York

Lyn Paget—Health Policy Partners

Ahmed Calvo—Health Resources and Services Administration

Ian Corbridge—Health Resources and Services Administration

Chris DeGraw—Health Resources and Services Administration

Leonard Epstein—Health Resources and Services Administration

Reem Ghandour—Health Resources and Services Administration

Seiji Hayashi—Health Resources and Services Administration

Sarah Linde-Feucht—Health Resources and Services Administration

Michael Lu—Health Resources and Services Administration

Samantha Meklir—Health Resources and Services Administration

Andrew Roszak—Health Resources and Services Administration

Mary Wakefield—Health Resources and Services Administration

John Seibel—HealthInsight New Mexico

Juliana Preston—HealthInsight Utah

Beth Averbeck—HealthPartners

David Gesko—HealthPartners

George Isham—HealthPartners

Thomas Kottke—HealthPartners

Thomas Von Sternberg—HealthPartners

Rick Luetkemeyer—HealthStrategy

Leslie Kelly Hall—Healthwise

Diane Limbo—Healthy Smiles for Kids of Orange County

John Pellicone—Helen Hayes Hospital

William Conway—Henry Ford Health System

Vanita Pindolia—Henry Ford Health System

Elizabeth Gilbertson—HEREIU Welfare Fund

Mary Blank—Highmark

Rubin Cohen—Hofstra University School of Medicine

June Lunney—Hospice and Palliative Nurses Association

Gail Austin Cooney—Hospice of Palm Beach County/Spectrum Health Inc.

Hayley Burgess—Hospital Corporation of America

Edward Septimus—Hospital Corporation of America

Louis Hoccheiser—Humana Inc.

Thomas James—Humana Inc.

Thomas James—Humana Inc.

Bryan Loy—Humana Inc.

Charles Stemple—Humana Inc.

Fredrik Tolin—Humana Inc.

Kyu Rhee—IBM

Mary Driscoll—Illinois Department of Public Health

Richard Snyder—Independence Blue Cross

Steve Udvarhelyi—Independence Blue Cross

Christopher Lamer—Indian Health Service

Steven Counsell—Indiana University School of Medicine

Floyd Fowler—Informed Medical Decision Making Foundation

Paula Graling—Inova Fairfax Hospital

Donald Goldmann—Institute for Healthcare Improvement

Sue Gullo—Institute for Healthcare Improvement

David Radley—Institute for Healthcare Improvement

Matthew Grissinger—Institute for Safe Medication Practices

Christina Michalek—Institute for Safe Medication Practices

Dolores Yanagihara—Integrated Healthcare Association

Allison Jackson—Intel

Barbara McCann—Interim HealthCare Inc.

Elizabeth Hammond—Intermountain Healthcare

Laura Heerman Langford—Intermountain Healthcare

Teri Kiehn—Intermountain Healthcare

Caterina Lasome—iON Informatics, LLC

Bob Russell—Iowa Department of Public Health

Meg Nugent—Iowa Healthcare Collaborative

Lance Roberts—Iowa Healthcare Collaborative

Nancy Zionts—Jewish Healthcare Foundation

Lisa Tripp—John Marshall Law School

Colleen Barry—Johns Hopkins Health System

Cynthia Boyd—Johns Hopkins Health System

Bruce Leff—Johns Hopkins Health System

Christoph Lehmann—Johns Hopkins Health System

Matthew McNabney—Johns Hopkins Health System

Robert Miller—Johns Hopkins Health System

Aaron Milstone—Johns Hopkins Health System

Lori Paine—Johns Hopkins Health System

Albert Wu—Johns Hopkins Health System

Patricia Abbott—Johns Hopkins University School of Nursing

David Domann—Johnson & Johnson Health Care Systems, Inc.

Christina Farup—Johnson & Johnson Health Care Systems, Inc.

Andy Amster—Kaiser Permanente

Amy Compton-Phillips—Kaiser Permanente

Douglas Corley—Kaiser Permanente

Sue Elam—Kaiser Permanente

Jamie Ferguson—Kaiser Permanente

Helen Lau—Kaiser Permanente

David Magid—Kaiser Permanente

Helene Martel—Kaiser Permanente

Ted Palen—Kaiser Permanente

David Pating—Kaiser Permanente

Elizabeth Paxton—Kaiser Permanente

Michael Schatz—Kaiser Permanente

Matt Stiefel—Kaiser Permanente

Jim Bellows—Kaiser Permanente

Jann Dorman—Kaiser Permanente

Elizabeth McGlynn—Kaiser Permanente

Lynn Searles—Kansas Department of Health and Environment

A.M. Barrett—Kessler Foundation

Bruce Pomeranz—Kessler Institute for Rehabilitation

Sean Muldoon—Kindred Healthcare

Laura Linebach—LA Care Health Plan

Rocco Ricciardi—Lahey Clinic Medical Center

Suma Thomas—Lahey Clinic Medical Center

Lauren Murray—Lamaze International

Paul Casale—Lancaster General Hospital

Cheryl Phillips—LeadingAge

Ian Chuang—Lockton Companies, LLC

Rebekah Gee—LSU School of Public Health

Anne Flanagan—Maine Department of Health

Elizabeth Mitchell—Maine Health Management Coalition

Ted Rooney—Maine Quality Counts

Scott Berns—March of Dimes

Cynthia Pellegrini—March of Dimes

Amit Acharya—Marshfield Clinic

Renee Webster—Maryland Department of Health

Elizabeth Daake—Massachusetts Department of Health

Joseph Betancourt—Massachusetts General Hospital

Liliana Bordeianou—Massachusetts General Hospital

Raymond Chung—Massachusetts General Hospital

Timothy Ferris—Massachusetts General Hospital

Elizabeth Mort—Massachusetts General Hospital

Laura Riley—Massachusetts General Hospital

Laura Riley—Massachusetts General Hospital

Karen Sepucha—Massachusetts General Hospital

David Shahian—Massachusetts General Hospital

David Torchiana—Massachusetts General Physicians Organization

David Polakoff—MassHealth

Robert Cima—Mayo Clinic

Pamela Foster—Mayo Clinic

Raymond Gibbons—Mayo Clinic

Catherine Roberts—Mayo Clinic

Eric Tangalos—Mayo Clinic

Karlene Phillips—Mayo Clinic

Gary Wingrove—Mayo Clinic

Charles Denk—MCH Epidemiology Program

Ginny Meadows—McKesson Corporation

Caroline Doebbeling—MDwise

Nicholas Sears—MedAssets, Inc.

Linus Santo Tomas—Medical College of Wisconsin

Peter Havens—Medical College of Wisconsin and Froedtert Hospital

Dana King—Medical University of South Carolina

Gail Stuart—Medical University of South Carolina

Zahid Butt—Medisolv, Inc.

Charlotte Alexander—Memorial Hermann Healthcare System

Roy Beasley—Memorial Hermann Healthcare System

M. Michael Shabot—Memorial Hermann Healthcare System

Lourdes Cuellar—Memorial Hermann Healthcare System—TIRR

David Pfister—Memorial Sloan-Kettering Cancer Center

Cristie Travis—Memphis Business Group on Health

Luther Clark—Merck & Co., Inc

Jennifer Bailit—MetroHealth Medical Center

Robin Shivley—Michigan Department of Health, EMS, and Trauma Systems

Michael O'Toole—Midwest Heart Specialists, Ltd.

Collette Pitzen—Minnesota Community Measurement

Diane Rydrych—Minnesota Department of Health

Vallire Hooper—Mission Hospital

Karen Fields—Moffitt Cancer Center

Jason Adelman—Montefiore Medical Center

Daniel Labovitz—Montefiore Medical Center

Helen Haskell—Mothers Against Medical Error

Leslie Zun—Mount Sinai Hospital

Peter Elkin—Mount Sinai Medical Center

R. Sean Morrison—Mount Sinai School of Medicine

Sean Morrison—Mount Sinai School of Medicine

Andrew Snyder—National Academy for State Health Policy

Gail Hunt—National Alliance for Caregiving

David Stevens—National Association of Community Health Centers

Robert Pestronk—National Association of County & City Health Officials

Denise Love—National Association of Health Data Organizations

Jane Hooker—National Association of Public Hospitals and Health Systems

Vickie Sears—National Association of Public Hospitals and Health Systems

Bruce Siegel—National Association of Public Hospitals and Health Systems

Jill Steinbruegge—National Association of Public Hospitals and Health Systems

Joan Zlotnik—National Association of Social Workers

Charles Moseley—National Association of State Directors of Developmental Disabilities Services

Martha Roherty—National Association of States United for Aging and Disabilities

Colleen Bruce—National Business Coalition on Health

Andrew Webber—National Business Coalition on Health

Dennis White—National Business Coalition on Health

Penney Berryman—National Business Group on Health

Helen Darling—National Business Group on Health

Pamela Kalen—National Business Group on Health

Sarah Brown—National Campaign to Prevent Teen and Unplanned Pregnancy

Steven Clauser—National Cancer Institute

Suzanne Heurtin-Roberts—National Cancer Institute

Linda Kinsinger—National Center for Health Promotion and Disease Prevention

Carol Allred—National Coalition for Women with Heart Disease

Mary Barton—National Committee for Quality Assurance

Margaret O'Kane—National Committee for Quality Assurance

Aldo Tinoco—National Committee for Quality Assurance

Phyllis Torda—National Committee for Quality Assurance

Michael Lardiere—National Council for Community Behavioral Healthcare

Nancy Whitelaw—National Council on Aging

Howard Kirkwood—National EMS Management Association

Keith Mason—National Forum for Heart Disease and Stroke Prevention

Brad Finnegan—National Governors Association

Marcia Thomas-Brown—National Health IT Collaborative for the Underserved

Leonardo Cuello—National Health Law Program

Deborah Reid—National Health Law Program

Mara Youdelman—National Health Law Program

Elena Rios—National Hispanic Medical Association

Carol Spence—National Hospice and Palliative Care Organization

Charles Homer—National Initiative for Children's Healthcare Quality

Jennifer Ustianov—National Initiative for Children's Healthcare Quality

Michael Lauer—National Institutes of Health

Marcel Salive—National Institutes of Health

Salina Waddy—National Institutes of Health

Adam Burrows—National PACE Association

Peter Schmidt—National Parkinson Foundation, Inc.

Tanya Alteras—National Partnership for Women & Families

Christine Bechtel—National Partnership for Women & Families

Debra Ness—National Partnership for Women & Families

Lee Partridge—National Partnership for Women & Families

Eva Powell—National Partnership for Women & Families

Kalahn Taylor-Clark—National Partnership for Women & Families

Janet Corrigan—National Quality Forum

Floyd Eisenberg—National Quality Forum

Laura Miller—National Quality Forum

Brock Slabach—National Rural Health Association

Robert Robin—Native Americans for Community Action, Inc.

Kathryn Blake—Nemours Foundation

Stephen Lawless—Nemours Foundation

Raj Sheth—Nemours Foundation

Mary Ann Clark—Neocure Group

Harold Miller—Network for Regional Healthcare Improvement

Bobbette Bond—Nevada Healthcare Policy Group LLC

Jay Kvam—Nevada State Health Division

Jose Montero—New Hampshire Department of Health and Human Services

Christine Stearns—New Jersey Business & Industry Association

Margaret Lumia—New Jersey Department of Health and Senior Services

David Knowlton—New Jersey Health Care Quality Institute

Ann Marie Sullivan—New York City Health and Hospitals Corporation

Eliot Lazar—New York Presbyterian Healthcare System

Harold Pincus—New York Presbyterian Healthcare System

Hussein Tahan—New York Presbyterian Healthcare System

Foster Gesten—New York State Department of Health

Norman Otsuka—New York University Hospital for Joint Diseases

Madeline Naegle—New York University, American Nurses Association

J. Emilio Carrillo—New York-Presbyterian Community Health Plan

Scott MacLean—Newton-Wellesley Hospital

Gregory Kapinos—North Shore-Long Island Jewish Health System

Louis Potters—North Shore-Long Island Jewish Health System

Kristofer Smith—North Shore-Long Island Jewish Health System

Jeffrey Susman—Northeast Ohio Medical University

William Rich—Northern Virginia Ophthalmology Associates

David Baker—Northwestern University

Romana Hasnain-Wynia—Northwestern University

David Stumpf—Northwestern University

Jane Sullivan—Northwestern University Feinberg School of Medicine

Mark Williams—Northwestern University Feinberg School of Medicine

Mary Jean Schumann—Nursing Alliance for Quality Care

Russell Leftwich—Office of eHealth Initiatives, State of Tennessee

Frank Johnson—Office of Employee Health & Benefits, State of Maine

Stephanie Mika—Office of the Assistant Secretary for Planning & Evaluation, HHS

Thomas Tsang—Office of the Governor, Hawaii

Jesse James—Office of the National Coordinator for Health Information Technology

Kevin Larsen—Office of the National Coordinator for Health Information Technology

Jacob Reider—Office of the National Coordinator for Health Information Technology

Joshua Seidman—Office of the National Coordinator for Health Information Technology

Allen Traylor—Office of the National Coordinator for Health Information Technology

Kaliyah Shaheen—Ohio Department of Health

Bernadette Melnyk—Ohio State University

Susan Moffatt-Bruce—Ohio State University

Michael Sayre—Ohio State University

Patrick Ross—Ohio State University Comprehensive Cancer Center—James Cancer Hospital

Gerene Bauldoff—Ohio State University, School of Nursing

Douglas Nee—OptiMed,Inc.

Mark Leenay—OptumHealth

Michael Lieberman—Oregon Health and Sciences University

Sydney Edlund—Oregon Patient Safety Commission

Roger Herr—Outcome Concept Systems

Kate Chenok—Pacific Business Group on Health

Emma Hoo—Pacific Business Group on Health

David Hopkins—Pacific Business Group on Health

Jennifer Huff—Pacific Business Group on Health

William Kramer—Pacific Business Group on Health

Seena Haines—Palm Beach Atlantic University

Paul Tang—Palo Alto Medical Foundation

Sue Pickens—Parkland Health & Hospital System

Michael Mirro—Parkview Health

Blackford Middleton—Partners HealthCare System, Inc.

Jason Spangler—Partnership for Prevention

Lori Frank—Patient Centered Outcomes Research Institute

Marci Nielsen—Patient Centered Primary Care Collaborative

Ron Stock—PeaceHealth Oregon Region

Chris Snyder—Peninsula Regional Medical Center

Peter Dillon—Penn State Hershey Medical Center

Michael Doering—Pennsylvania Patient Safety Authority

Eileen Kennedy—Pepco Holdings, Inc

Michael Ibara—Pfizer

Eleanor Perfetto—Pfizer

Laura Cranston—Pharmacy Quality Alliance

Kathleen Brady—Philadelphia Department of Public Health

Tina Cronin—Piedmont Medical Center

Susan Frampton—Planetree

Michael Lepore—Planetree

Richard Bankowitz—Premier healthcare alliance

Gina Pugliese—Premier healthcare alliance

Dennis Kaldenberg—Press Ganey Associates

Larry Cohen—Prevention Institute

James Lee—Providence Everett Medical Center

Robert Hellrigel—Providence Health & Services

Ron Bialek—Public Health Foundation

Mary Pittman—Public Health Institute

Louis Diamond—QHC Advisory Group, LLC

Dawn Fitzgerald—Qsource

Sharon Hibay—Quality Insights of Pennsylvania

Bonnie Paris—Quality Quest for Health of Illinois

David Seidenwurm—Radiological Associates of Sacramento Medical Group, Inc.

Leona Cuttler—Rainbow Babies and Children's Hospital

Arthur Kellermann—RAND Corporation

Debra Saliba—RAND Corporation

Kathleen Aller—Recommind, Inc.

Mary Van de Kamp—RehabCare

Darlene Skorski—Rhode Island Department of Health—Office of Facilities Regulation

David Krol—Robert Wood Johnson Foundation

Carey Smoak—Roche Laboratories, Inc.

Stephen Edge—Roswell Park Cancer Institute

Kathleen Lohr—RTI International

Ruth Kleinpell—Rush University Medical Center

Shannon Sims—Rush University Medical Center

Victoria Nahum—Safe Care Campaign

James Dunford—San Diego Fire-Rescue

Paul Merguerian—Seattle Children's Hospital

Rita Mangione-Smith—Seattle Children's Research Institute

Charissa Raynor—Service Employees International Union

Dale Shaller—Shaller Consulting Group

Karen Nielsen—Siemens Medical Solutions USA

J. Marc Overhage—Siemens Medical Solutions USA

Christopher Smiley—Smiley Family Dentistry, PC

Richard Bringewatt—SNP Alliance

William Grobman—Society for Maternal-Fetal Medicine

Kate Menard—Society for Maternal-Fetal Medicine

Mitchell Levy—Society of Critical Care Medicine

Janet Nagamine—Society of Hospital Medicine

Wendy Nickel—Society of Hospital Medicine

Howard Barnebey—Specialty Eyecare Centre

Jerad Widman—Spring Hill Family Medicine

Dennis Rivenburgh—St Anthony's

Mohamad Fakih—St. John Hospital and Medical Center

Kathleen Rice Simpson—St. John's Mercy Health Care

Joseph Laver—St. Jude Children's Research Hospital

Louise Probst—St. Louis Area Business Health Coalition

Mark Sanz—St. Patrick Hospital

Risha Gidwani—Stanford University Medical Center

John Morton—Stanford University Medical Center

Marc Leib—State of Arizona Medicaid Program

Ruth Leslie—State of New York Department of Health

John Maese—Staten Island University Hospital

Bruce Auerbach—Sturdy Memorial Hospital

Amina Chaudhry—Substance Abuse and Mental Health Services Administration

Frances Cotter—Substance Abuse and Mental Health Services Administration

Pamela Hyde—Substance Abuse and Mental Health Services Administration

Rita Vandivort-Warren—Substance Abuse and Mental Health Services Administration

Thomas File—Summa Health System

Tina Picchi—Supportive Care Coalition

Lois Cross—Sutter Health

A. John Blair—Taconic IPA, Inc.

Chad Bennett—Telligen

Julie Kuhle—Telligen

Liz Johnson—Tenet Healthcare Corporation

Ann Reed—Tennessee Department of Health

William Glomb—Texas Health and Human Services Commission

Dennis Andrulis—Texas Health Institute

Steven Brotman—The Advanced Medical Technology Association

Cheryl DeMars—The Alliance

Mark McClellan—The Brookings Institute

Anne-Marie Audet—The Commonwealth Fund

Mary Jane Koren—The Commonwealth Fund

Eugene Nelson—The Dartmouth Institute

Jesse Pines—The George Washington University Medical Center

Gerard Castro—The Joint Commission

Mark Chassin—The Joint Commission

Patricia Craig—The Joint Commission

Patricia Kurtz—The Joint Commission

Jerod Loeb—The Joint Commission

Crystal Riley—The Joint Commission

Heather Sherman—The Joint Commission

Margaret VanAmringe—The Joint Commission

Ann Watt—The Joint Commission

Susan Yendro—The Joint Commission

Leah Binder—The Leapfrog Group

Barbara Rudolph—The Leapfrog Group

Nadine Gracia—The Office of Minority Health

Mady Chalk—Treatment Research Institute

Paul Conlon—Trinity Health

Tami Mark—Truven Health Analytics

Randel Johnson—U.S. Chamber of Commerce

Salma Lemtouni—U.S. Food and Drug Administration

Philip Schoenfeld—UM Medical School

Jordan Eisenstock—UMass Memorial Medical Center

Devorah Rich—United Auto Workers Retiree Medical Benefits Trust

Rhonda Robinson Beale—United Behavioral Health

Barbara Corn—UnitedHealth Group

Rhonda Medows—UnitedHealth Group

Renae Stafford—University North Carolina

Alayne Markland—University of Alabama at Birmingham

Robert Weech-Maldonado—University of Alabama at Birmingham

Doug Campos-Outcalt—University of Arizona College of Medicine

Steven Chen—University of California Davis

Francis Lu—University of California Davis

Richard White—University of California Davis

Solomon Liao—University of California Irvine

Sherrie Kaplan—University of California Irvine School of Medicine

John Kusske—University of California Irvine School of Medicine

Nasim Afsar-manesh—University of California Los Angeles

Jim Crall—University of California Los Angeles

Bonnie Zima—University of California Los Angeles Center for Health Services & Society

Christopher Saigal—University of California Los Angeles Medical Center

Theodore Ganiats—University of California San Diego

Charlene Harrington—University of California San Francisco

Louise Walter—University of California San Francisco

Nancy Donaldson—University of California San Francisco School of Nursing

Marshall Chin—University of Chicago

William McDade—University of Chicago

William Dale—University of Chicago Medical Center

Nancy Lowe—University of Colorado Denver

Mark Metersky—University of Connecticut Health Center

Ramon Bautista—University of Florida HSC/Jacksonville

Tim Williamson—University of Kansas Medical Center

Katherine Reeder—University of Kansas School of Nursing

Judith Warren—University of Kansas School of Nursing

Joanna Sikkema—University of Miami, School of Nursing and Health Studies

William Barsan—University of Michigan Hospitals and Health Centers

James Carpenter—University of Michigan Hospitals and Health Centers

Elaine Chottiner—University of Michigan Hospitals and Health Centers

Curtis Collins—University of Michigan Hospitals and Health Centers

Karen Farris—University of Michigan Hospitals and Health Centers

Ella Kazerooni—University of Michigan Hospitals and Health Centers

Janet Larson—University of Michigan Hospitals and Health Centers

Jean Malouin—University of Michigan Hospitals and Health Centers

Marc Moote—University of Michigan Hospitals and Health Centers

Anne Pelletier Cameron—University of Michigan Hospitals and Health Centers

Linda Lindeke—University of Minnesota Amplatz Children's Hospital

Ira Moscovice—University of Minnesota Rural Health Research Center

Kristi Anne Henderson—University of Mississippi Medical Center

Bonnie Wakefield—University of Missouri

John Fildes—University of Nevada Las Vegas Medical Center

Ethan Basch—University of North Carolina at Chapel Hill

Jessica Lee—University of North Carolina at Chapel Hill

Sidney Smith—University of North Carolina at Chapel Hill

David Weber—University of North Carolina at Chapel Hill

Lynn Wegner—University of North Carolina School of Medicine

Lawrence Marks—University of North Carolina, School of Medicine

Dale Bratzler—University of Oklahoma Health Sciences Center

Mark Wolraich—University of Oklahoma Health Sciences Center

Judith Hibbard—University of Oregon

Leah Marcotte—University of Pennsylvania

Brendan Carr—University of Pennsylvania Health System

Lee Fleisher—University of Pennsylvania Health System

Jerry Johnson—University of Pennsylvania Health System

Frank Leone—University of Pennsylvania Health System

David Casarett—University of Pennsylvania School of Medicine

Kathryn Bowles—University of Pennsylvania School of Nursing

Nancy Hanrahan—University of Pennsylvania School of Nursing

Therese Richmond—University of Pennsylvania, School of Nursing

Douglas White—University of Pittsburgh

Donald Yealy—University of Pittsburgh Medical Center

Carl Sirio—University of Pittsburgh School of Medicine

Heidi Donovan—University of Pittsburgh School of Nursing

Laurent Glance—University of Rochester

Kevin Fiscella—University of Rochester School of Medicine

Jeffrey Beal—University of South Florida

Barbara Turner—University of Texas Health Science Center at San Antonio

Eduardo Bruera—University of Texas MD Anderson Cancer Center

Kenneth Ottenbacher—University of Texas Medical Branch at Galveston

Ethan Halm—University of Texas Southwestern Medical Center

Mambarambath Jaleel—University of Texas Southwestern Medical Center

Kathy Rinnert—University of Texas Southwestern Medical Center

Craig Rubin—University of Texas Southwestern Medical School

Victoria Jordan—University of Texas-MD Anderson Cancer Center

John Skibber—University of Texas-MD Anderson Cancer Center

Barbara Summers—University of Texas-MD Anderson Cancer Center

Ronald Walters—University of Texas-MD Anderson Cancer Center

Amy Hessel—University of Texas-MD Anderson Medical Center

Paul Glassman—University of the Pacific School of Dentistry

David Classen—University of Utah School of Medicine

Michael Farber—University of Vermont College of Medicine

Pamela Cipriano—University of Virginia Health System

Rachel Grob—University of Wisconsin Center for Patient Partnerships

Elizabeth Jacobs—University of Wisconsin, Department of Medicine

Patricia Brennan—University of Wisconsin-Madison

Tracy Schroepfer—University of Wisconsin-Madison

Christine Hunter—US Office of Personnel Management

John O'Brien—US Office of Personnel Management

Iona Thraen—Utah Department of Health

Jim Smith—Utica College

David Penson—Vanderbilt University Medical Center

W. Stuart Reynolds—Vanderbilt University Medical Center

Peter Almenoff—Veterans Health Administration

Caroline Blaum—Veterans Health Administration

John Duda—Veterans Health Administration

Stephan Fihn—Veterans Health Administration

Joseph Francis—Veterans Health Administration

Vivienne Halpern—Veterans Health Administration

Marcia Insley—Veterans Health Administration

Michael Kelley—Veterans Health Administration

Daniel Kivlahan—Veterans Health Administration

Robert Petzel—Veterans Health Administration

Patricia Quigley—Veterans Health Administration

Scott Shreve—Veterans Health Administration

Patricia Sinnott—Veterans Health Administration

Donna Washington—Veterans Health Administration

Edward Gill—Virginia Commonwealth University Medical Center

Cathie Furman—Virginia Mason Medical Center

Johannes Koch—Virginia Mason Medical Center

Jolynn Suko—Virginia Mason Medical Center

Carol Mullin—Virtua Health

Margaret Terry—Visiting Nurse Associations of America

Carol Raphael—Visiting Nurse Service of New York

Robert Rosati—Visiting Nurse Service of New York

William Frohna—Washington Hospital Center

Linda Furkay—Washington State Department of Health

David Mancuso—Washington State Department of Social & Health Services

Jeffery Thompson—Washington State Medicaid

Michael Kaplitt—Weill Cornell Medical College

Aron Halfin—WellPoint

Richard Hastreiter—WellPoint

Jennifer Malin—WellPoint

Sarah Sampsel—WellPoint

Grace Ting—WellPoint

Tracy Wang—WellPoint

Alonzo White—WellPoint

Christy Whetsell—West Virginia University Hospitals

Frank Ghinassi—Western Psychiatric Institute & Clinic of the University of Pittsburgh Medical Center

Lori Nichols—Whatcom Health Information Network

Christopher Queram—Wisconsin Collaborative for Healthcare Quality

John Bott—Wisconsin Department of Employee Trust Funds

Lois Sater—Wisconsin Division of Public Health

Nancy Faller—Wound, Ostomy and Continence Nurses Society

Jeptha Curtis—Yale New Haven Health System

Elizabeth Drye—Yale New Haven Health System

Marcella Nunez-Smith—Yale New Haven Health System

Patrick O'Connor—Yale New Haven Health System

Mary Tinetti—Yale New Haven Health System

Patricia Button—Zynx Health

David Rhew—Zynx Health

Appendix F: National Quality Forum—Background

Despite the hard work of many, there is broad recognition that our healthcare system can do a better job on quality, safety, and affordability. This reality, in the context of a cost-conscious economy, has re-energized a national commitment to simultaneously improve care and responsibly constrain healthcare costs. State leaders, local governments, a broad swath of federal healthcare agencies, and an increasing number of other public- and private-sector organizations that constitute the quality movement are at the center of that resurgence. NQF is a public service organization that helps unite all of these organizations in their pursuit to make healthcare better, safer, and affordable.

Established in 1999 as the standard-setting organization for healthcare performance measures, NQF today has a much-broadened mission to:

• Build consensus on national priorities and goals for performance improvement, and work in partnership with the public and private sectors to achieve them.
• Endorse and maintain best-in-class standards for measuring and publicly reporting on healthcare performance quality.
• Promote the attainment of national healthcare improvement goals and the use of standardized measures through education and outreach programs.

NQF is recognized as a voluntary consensus standard-setting organization under the National Technology Transfer and Advancement Act of 1995. Its process for reaching consensus adheres to the Office of Management and Budget's formal definition of consensus.³¹

The NQF Board of Directors governs the organization and is composed of 31 voting members—key public- and private-sector leaders who represent major stakeholders in America's healthcare system. Consumers and those who purchase healthcare hold a simple majority of the at-large seats (see Appendix B). In 2012, NQF convened more than 800 hundred experts across every stakeholder group who contributed their time, experience, and insights to measure-review, measure-selection, and priority-setting committees (see Appendix E).

In recent years as part of a close working partnership with HHS, the variety of NQF-endorsed measures has greatly expanded to address most settings of care, conditions, and provider types. NQF's measure portfolio includes measures of clinical process, patient experience of care, the actual outcomes of care, the costs and resources that go into providing care, as well as select structural measures. The portfolio is being enhanced with advanced measures, such as patient-reported outcomes and cross-cutting care-coordination measures. At the same time, NQF carefully manages its portfolio to be lean, retiring measures that no longer meet the more rigorous criteria. In the past year alone, 430 measures were submitted to NQF and 301, or nearly 70 percent, were endorsed. This endorsement rate—or ratio of submitted to endorsed measures—reflects NQF's efforts to systematically raise the bar on performance measurement and to fill key measurement gap areas even as it aggressively seeks to reduce the burden on providers by eliminating duplicative measures that add unnecessary data collection and administrative workload.

To be NQF endorsed, a measure must capture a process or outcome that is important to measure and report, be scientifically acceptable, be feasible to collect, and provide useful results. NQF conducts an eight-step, consensus-based process for reviewing measures and other standards; this process has been continually improved over a decade, and is as follows:

1. Call for Nominations allows anyone to suggest a candidate for the committee that will oversee the project. Committees are diverse, often encompassing experts in a particular field, providers, scientists, and consumers. After selection, NQF posts committee rosters on its Web site to solicit public comments on the composition of the panel and makes adjustments as needed to ensure balanced representation.

2. Call for Measures starts a 30-day period for developers to submit a measure or practice through NQF's online submission forms.

3. Steering Committee Review puts submitted measures to a four-part test to ensure they reflect sound science, will be useful to providers and patients, and will make a difference in improving quality. The expert steering committee conducts this detailed review in open sessions, each of which starts a limited period for public comment.

4. Public Comment solicits input from anyone who wishes to respond to a draft report that outlines the steering committee's assessment of measures for possible endorsement. The steering committee may request a revision to the proposed measures.

5. Member Vote asks NQF members to review the draft report and cast their votes on the endorsement of measures.

6. CSAC Review marks the point at which the NQF Consensus Standards Approval Committee (CSAC) deliberates on the merits of the measure and the issues raised during the review process, and makes a recommendation on endorsement to the Board of Directors. The CSAC includes consumers, purchasers, healthcare professionals, and others. It provides the big picture to ensure that standards are being consistently assessed from project to project.

7. Board Ratification asks for review and ratification by the NQF Board of Directors of measures recommended for endorsement.

8. Appeal opens a period when anyone can appeal the Board's decision.

Review committees comprise multiple stakeholders; consumer organizations and individual patients are equal partners with clinicians and other stakeholders throughout the process. There is a strong commitment to transparency: NQF invites public participation at every step, ranging from nominations for committees to comments and votes on specific measures. Endorsed measures are re-evaluated every three years to ensure their continuing relevance with current science and their actual use and usefulness in the field, and to determine whether they continue to represent the best in class compared to new measures. At any time, NQF can also conduct an ad hoc review of a measure if there is evidence of unintended consequences related to measurement or emerging clinical evidence that should result in a change to the measure.

Measures included in the NQF portfolio are developed and maintained by about 65 different organizations including the Centers for Medicare and Medicaid Services (CMS), the National Committee for Quality Assurance (NCQA), the Physician Consortium for Performance Improvement, convened by the American Medical Association (AMA-PCPI), Ingenix, The Joint Commission, American College of Surgeons (ACS), Bridges to Excellence, Cleveland Clinic, Minnesota Community Measurement, and Pharmacy Quality Alliance.

Many public- and private-sector leaders contributed to developing NQF's multi-stakeholder consensus process in the measure-endorsement realm. In recognition of this unique public service, HHS is required under statute to contract with a consensus-based entity, and contracted with NQF to convene diverse stakeholder groups to advise the public sector on priorities for healthcare improvement, related implementation strategies, and selection of measures to both drive these strategies and gauge results. The NQF-convened NPP and MAP and their published reports are tangible outcomes of this work. An equally important outcome of these partnerships is the ongoing alignment across stakeholder groups and across public- and private-sector leaders about which levers are most powerful in both improving healthcare performance and making the delivery system more patient centered.

NQF was initially funded primarily through grants from major philanthropic foundations, including the Robert Wood Johnson Foundation and the Commonwealth Fund. NQF in turn built a strong membership base across all those who care about advancing healthcare quality; membership dues continue to provide annual funding for NQF's work.

In 2012, NQF received $4.43 million a year in membership dues, an amount equaling 18 percent of its total budget. When combined with private foundation funding, 23 percent of NQF's budget comes from the private sector, with the remainder of its funding stemming from the public sector. In addition, the value of uncompensated donated time in 2012—some 55,000 hours of work done on a volunteer basis by healthcare leaders and experts—is conservatively estimated to equal another $4 million in private funding for NQF's work. Scaling up NQF's capacity became a necessity when the public sector, in its role as the largest American healthcare purchaser, made a serious commitment to buying healthcare based on value. This policy direction immediately generated the need for a more sustainable, steady resource that stood ready to regularly review and endorse performance measures.

NQF has been fortunate to have received support from the federal government for more than 10 years, particularly since 2008 when federal leaders strongly committed themselves to designing and implementing a value-driven agenda for healthcare. More specifically:

• MIPPA has provided NQF with $10 million annually over a four-year period starting in 2009, which was extended for FY 2013 by HR8 (PL 112-240). These funds—awarded to NQF through a competitive process—support the organization's efforts to identify priority areas for improvement, endorse and update related performance measures, foster the transition to an electronic environment, and report annually to Congress on the status and progress to date of this effort.

ACA has provided NQF with support of about $10 million annually, starting in 2011. Under Section 3014, Congress directed HHS to contract with “the consensus-based entity under contract” to provide multi-stakeholder input into the NQS, as well as input to the Secretary of HHS on the selection of measures for use in various quality programs that utilize the federal rulemaking process for measure selection.

IV. Secretarial Comments on the Annual Report to Congress

This 2013 Annual Report describes NQF's work in 2012 to fulfill the requirements specified in section 1890 of the Social Security Act. This section of the Social Security Act requires the Secretary of the Department of Health and Human Services to “have in effect a contract with a consensus-based entity, such as the National Quality Forum,” to perform certain duties including those related to performance measurement and NQS priorities. The Social Security Act also requires by not later than March 1 of each year (beginning with 2009), that the CBE shall submit to Congress and the Secretary of the Department of Health and Human Services a report containing a description of:

(i) Implementation of quality and efficiency measurement initiatives under the Social Security Act and the coordination of such initiatives with quality and efficiency initiatives implemented by other payers;

(ii) recommendations on an integrated national strategy and priorities for health care performance measurement;

(iii) performance of its duties required under its contract with HHS;

(iv) gaps in endorsed quality and efficiency measures, and where quality and efficiency measures are unavailable or inadequate to identify or address such gaps;

(v) areas in which evidence is insufficient to support endorsement of quality and efficiency measures in priority areas identified by the Secretary under the national strategy and where targeted research may address such gaps; and

(vi) convening multi-stakeholder groups to provide input on: 1) The selection of quality and efficiency measures for use in various Medicare programs, in reporting performance information to the public; and in other health care programs; and 2) national priorities for improvement in population health and the delivery of health care services for consideration under the national quality strategy.

This 2013 report fulfills the statutory requirement for the annual report described above and describes the results of work that NQF, as the CBE, undertook in 2012.

For example, in 2012, NQF managed its portfolio of more than 700 endorsed measures by replacing some measures with improved measures; removing measures that were no longer effective or where the evidence base had evolved; and expanding the portfolio to address well-recognized measurement gaps. NQF reviewed 430 submitted measures and endorsed 301 of them. This set included 81 new measures and 220 measures that maintained their endorsement after being considered in light of new evidence and/or against new competing measures submitted to NQF for consideration. The newly endorsed measures align with needs identified in the NQS and address several critical areas, including patient outcomes, underserved populations, healthcare disparities, and hospital readmissions.

In 2012, NQF's National Priorities Partnership (NPP), a collaborative public-private partnership, focused on how to advance patient safety by aligning its work with HHS' “Partnership for Patients” initiative. Through a series of web-based and in-person meetings, nearly 2,700 participants from multiple sectors learned about and shared new improvement approaches, information, tools, and professional connections to improve health care safety. The NPP also developed action plans to focus a range of national and local organizations in diverse sectors on how to align efforts to reduce preventable readmissions and improve maternity care, and created a web-based “action registry” to track improvement activities focused on readmissions and maternity care to enable learning across participants. Launched in the fourth quarter of 2012, by March 2013, the registry housed over 50 actions by 30 different organizations.

In 2012, NQF also continued its work to facilitate the electronic reporting of quality measures using electronic health records (EHRs) that health care providers across the nation are adopting. NQF's work on these “eMeasures” included standardizing data elements so the same quality of care information can be collected from different EHRs. NQF also convened an eMeasure Learning Collaborative to help multiple parties address barriers to developing and implementing eMeasures.

NQF's Measure Applications Partnership (MAP) provided multi-stakeholder input to HHS about the potential use of quality measures in more than 17 different Medicare quality reporting and performance programs and the Medicare and Medicaid Electronic Health Record (EHR) Incentive Program. This input was critical to HHS programs. At the same time, MAP released its Families of Measures report, which defined measure families in four key areas—safety, care coordination, cardiovascular, and diabetes care—with the goal of promoting more cohesion and integration of care regardless of setting, provider, level of care intensity, or timing of care.

In 2012, NQF also conducted an analysis of its current measures portfolio against both the NQS priority areas and high-impact Medicare and child health conditions. This analysis found that while many NQF measures address patient safety, fewer measures address patient and family engagement. For example, measures of shared decision-making, patient navigation and self-management, healthy lifestyle behaviors, community interventions to improve health, and access, cost, and resource use are significantly less prevalent than safety measures. The analysis also found gaps in measures of preventive care, patient-reported outcomes (particularly quality of life and functional status), appropriateness (particularly for specialty care), access to timely palliative care, and health and healthcare disparities. Additionally, the analysis revealed the need for better population-level measures to assess improvements in health and healthcare. And, while certain high-impact conditions common to adults have an abundance of measures—e.g., cardiovascular disease, end-stage renal disease, and diabetes—many of the high-impact childhood conditions have few or no NQF-endorsed measures.

These and the other activities described in the Annual Report reflect the wide scope of work required for sound measurement of health care quality—and the accompanying hard work needed for the continued improvement of health care. HHS thanks NQF for its hard work and submission of this report.

V. Future Steps

The work reflected in this annual report was produced under HHS' initial four-year contract to NQF which was executed in 2009 and will expire in 2013.

To continue to fulfill the statutory requirement for a contract with a consensus-based entity, HHS competitively procured a new contract with NQF in September 2012. Through this new contract, NQF will continue to perform the statutory activities for the CBE described above in support of HHS' efforts to achieve the aims of the NQS— better care, healthier people and communities, and affordable care.

VI. Collection of Information Requirements

This document does not impose information collection and recordkeeping requirements. Consequently, it need not be reviewed by the Office of Management and Budget under the authority of the Paperwork Reduction Act of 1995 (44 U.S.C. 35)

¹ Measure Applications Partnership. Pre-Rulemaking Report: Input on Measures Under Consideration by HHS for 2012 Rulemaking. Washington, DC: National Quality Forum, 2013.

² National Quality Forum. NQF's Portfolio of Measures: Who is Using it, and how is it Evolving? Washington, DC: National Quality Forum, January 2012.

³ National Quality Forum. NQF Measure Portfolio Report. Washington, DC: National Quality Forum, November 2012.

⁴ Damberg CL, Sorbero ME, Lovejoy SL et al. An Evaluation of the Use of Performance measures in Health Care. Santa Monica, CA: RAND Corporation; 2011. Available at http://www.rand.org/content/dam/rand/pubs/technical_reports/2011/RAND_TR1148.pdf. Accessed December 2012.

⁵ Main E, Oshiro B, Chagolla B, Bingham D, Dang-Kilduff L, and Kowalewski L. Elimination of Non-medically Indicated (Elective) Deliveries Before 39 Weeks Gestational Age. (California Maternal Quality Care Collaborative Toolkit to Transform Maternity Care). Developed under contract #08-85012 with the California Department of Public Health; Maternal, Child and Adolescent Health Division; First edition published by March of Dimes, July 2010.

⁶ Childbirth Connection. Vaginal or Cesarean Birth: What Is at Stake for Women and Babies? New York: Childbirth Connection; 2012. Available at http://transform.childbirthconnection.org/reports/cesarean.

⁷ Sakala C, Corry MP. Evidence-Based Maternity Care: What It Is and What It Can Achieve. New York: Milbank Memorial Fund in collaboration with Childbirth Connection and Reforming States Group; 2008. Available at http://www.childbirthconnection.org/pdfs/evidence-based-maternity-care.pdf.

⁸ Jencks SF, Williams MV, Coleman EA, Rehospitalizations among patients in the Medicare fee-for-service program, New Engl J Med, 2009;360(14):1420-1421.

⁹ Pennsylvania Patient Safety Advisory. Leveraging healthcare policy changes to decrease hospital 30-day readmission rates, Pa Patient Saf Advis, 2010 March;7(1):1-8.

¹⁰ Medicare Payment Advisory Commission. Report to Congress: Promoting Greater Efficiency in Medicare. Washington, DC; 2007. Pp. 103-199.

¹¹ Saliba D, Kington R, Buchanan J, et al., Appropriateness of the decision to transfer nursing facility residents to hospital, J Am Geriatr Soc, 2000;48:154-163.

¹² Medicare Payment Advisory Commission (MEDPAC). Report to the Congress: Reforming the Delivery System. Washington, DC: MedPAC; 2008. Available at http://medpac.gov/documents/Jun08_EntireReport.pdf. Accessed October 2011.

¹³ Institute of Medicine. To Err is Human. Washington, DC: National Academies Press; 2001.

¹⁴ Banks J, et al., Disease and disadvantage in the United States and in England, JAMA, 2006;295(17):2037-2045.

¹⁵ Hoyert DL, et al., Annual summary of vital statistics: 2004, Pediatrics, 2006; 117(1):168-183.

¹⁶ Weiss JE, Mushinski M, International mortality rates and life expectancy: selected countries, Statistical Bulletin—Metropolitan Life Insurance Company, 1999;80(1):13-21.

¹⁷ Department of Health and Human Services (HHS), Office of the Assistant Secretary for Health (ASH), Initiatives, Washington, DC: HHS, ASH: 2011. Available at http://www.hhs.gov/ophs/initiatives/mcc/index.html. Last accessed December 2011.

¹⁸ Thorpe KE, Howard DH, The rise in spending among Medicare beneficiaries: the role of chronic disease prevalence and changes in treatment intensity, Health Aff, 2006;25(5):w378-w388.

¹⁹ Gijsen R, Hoeymans N, Schellevis FG, et al., Causes and consequences of comorbidity: a review, J Clin Epidemiol, 2001;54(7):661-674.

²⁰ Boult C, Wieland GD, Comprehensive primary care for older patients with multiple chronic conditions: “nobody rushes you through”, JAMA, 2010;304(17):1936-1943.

²¹ Parekh AK, Barton MB, The challenge of multiple comorbidity for the US health care system, JAMA, 2010;303(13):1303-1304.

²² Wolff JL, Starfield B, Anderson G, Prevalence, expenditures, and complications of multiple chronic conditions in the elderly, Arch Intern Med, 2002;162(20):2269-2276.

²³ Boyd CM, Boult C, Shadmi E, et al., Guided care for multimorbid older adults, Gerontologist, 2007;47(5):697-704.

²⁴ Institute of Medicine (IOM). Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Washington, DC: National Academies Press; 2003. Available at http://www.nap.edu/openbook. php?isbn=030908265X. Last accessed August 2012.

²⁵ Measure Applications Partnership. Pre-Rulemaking Report: Input on Measures Under Consideration by HHS for 2012 Rulemaking. Washington, DC: National Quality Forum, 2013.

²⁶ Measure Applications Partnership. Pre-Rulemaking Report: Input on Measures Under Consideration by HHS for 2012 Rulemaking. Washington, DC: National Quality Forum, 2013.

²⁷ Damberg CL, Sorbero ME, Lovejoy SL et al. An Evaluation of the Use of Performance measures in Health Care. Santa Monica, CA: RAND Corporation; 2011. Available at http://www.rand.org/content/dam/rand/pubs/technical_reports/2011/RAND_TR1148.pdf. Accessed December 2012.

²⁸ National Quality Forum. NQF's Portfolio of Measures: Who is Using it, and how is it Evolving? Washington, DC: National Quality Forum, January 2012.

²⁹ National Quality Forum. NQF Measure Portfolio Report. Washington, DC: National Quality Forum, November 2012.

³⁰ See rwjf.org/en/about-rwjf/newsroom/interactives/71857.html.

³¹ The White House, U.S. Office of Management and Budget (OMB). Circular No. A-119, February 10, 1998. Washington, DC: OMB; 1998. Available at www.whitehouse.gov/omb/circulars_a119/. Last accessed January 2012.