. Author manuscript; available in PMC: 2015 Aug 15.
Published in final edited form as:Cancer. 2014 Aug 15;120(0 16):2575–2583. doi:
10.1002/cncr.28821 AbstractThe National Breast and Cervical Cancer Early Detection Program (NBCCEDP) of the Centers for Disease Control and Prevention (CDC) is implemented through cooperative agreements with state health departments, US territories, and tribal health organizations (grantees). Grantees typically contract with clinicians and other providers to deliver breast and cervical cancer screening and diagnostic services. As required by the CDC, grantees report biannually a subset of patient and clinical level program data known as the Minimum Data Elements. Rigorous processes are in place to ensure the completeness and quality of program data collection. In this article, the authors describe the NBCCEDP data-collection processes and data management system and discusses how data are used for 1) program monitoring and improvement, 2) evaluation and research, and 3) policy development and analysis. They also provide 2 examples of how grantees use data to improve their performance.
Keywords: screening, data management, public health, quality measures, program evaluation
INTRODUCTIONIn 2009, 211,731 US women were diagnosed with breast cancer, and 40,676 women died from the disease. In addition, 12,357 women were diagnosed with, and 3909 died from, cervical cancer.1 Research results have demonstrated that screening is an effective prevention strategy to reduce the morbidity and mortality of several cancers, including breast and cervical cancer.2 Currently, the US Preventive Services Task Force (USPSTF) recommends mammography screening every 2 years for women ages 50 to 74 years and Papanicolaou (Pap) testing every 3 years for women ages 21 to 65 years with the option of converting to Pap testing combined with human papillomavirus (HPV) testing when a woman turns age 30 years.3 Despite screening recommendations, it has been observed that disparities in breast and cervical cancer death rates are especially high in certain racial/ethnic groups and in medically underserved populations because of factors such as barriers to early detection and screening, lack of medical insurance coverage, and inadequate access to cancer treatment.4
To improve breast and cervical cancer screening among medically underserved women, Congress authorized the Centers for Disease Control and Prevention (CDC) to develop the National Breast and Cervical Cancer Early Detection Program (NBCCEDP) in 1991 under the Breast and Cervical Cancer Mortality Prevention Act passed in1990 (Public Law 101–354).5 The NBCCEDP is a comprehensive public health program that helps low-income, underinsured, and uninsured women gain access to breast and cervical screening and diagnostic services. The program is implemented through cooperative agreements awarded to states, tribes, and US territories. The CDC estimated that more than 22,000 local providers had participated in the program as of 2002.6 Grantees typically contract with a broad range of clinical providers to deliver screening and diagnostic services. Through the hard work of dedicated national partners; state, tribal, and territorial health officials; community leaders; and medical care providers, the NBCCEDP has provided more than 11 million breast and cervical cancer screening and diagnostic tests to more than 4.5 million low-income, underserved, and uninsured women.7 The NBCCEDP is more fully described elsewhere in this supplemental issue of Cancer.8
NBCCEDP grantees not only provide screening and diagnostic services to women, but they collect and analyze data as part of their program evaluation, which the CDC has identified as an essential component of a public health program.9 Data from the NBCCEDP provide a rich source of information, permitting the CDC to ensure accountability, monitor program outcomes, support planning, improve the performance of grantees, respond to stakeholder inquiries, and refine NBCCEDP policies.
In this article, we describe the NBCCEDP data-collection processes and data management system and discuss how NBCCEDP data are used for 1) program monitoring and improvement, 2) evaluation, and 3) policy development and analysis. We also provide 2 examples of how grantees use the collected data to improve their program performance and management.
The NBCCEDP Data SystemData management and evaluation are distinct program components of the NBCCEDP. To ensure the provision of quality screening services and follow-up of the women screened, grantees are required to collect data on the services provided and report a standardized subset of data known as the Minimum Data Elements (MDEs) to the CDC. The MDEs describe the demographic characteristics and clinical outcomes of women enrolled in the NBCCEDP. The CDC prioritizes the use of these data and makes a significant investment in data management. An economic analysis of 9 grantees’ NBCCEDP activity-based costs conducted in 2005 revealed that 10.9% of their annual resources were allocated to data management.10
Grantees report a set of nearly 100 standardized data elements that are considered minimally necessary to monitor demographics and clinical outcomes of women who receive services through the NBCCEDP. The MDEs record includes standardized information on each encounter provided through the program; it also includes unique patient identifiers that can be used to track screening and diagnostic services to individual women over time. To comply with patient-protection policies, grantees obtain written patient consent for data collection and use and ensure confidentiality. Table 1 summarizes the primary MDE reporting categories. A multidisciplinary committee within the CDC Division of Cancer Prevention and Control (DCPC), along with the data management contractor, advises NBCCEDP grantees on MDE data requirements, reporting, and standards.
TABLE 1.National Breast and Cervical Cancer Early Detection Program Minimum Data Elements
Category Data Item Category Data Item Enrollment location Grantee screening program Cervical diagnostic procedures Colposcopy without biopsy County of screening Colposcopy-directed biopsy Enrollment site Loop electrosurgical excision procedure (LEEP) Patient/record identification Patient identifier Cold knife cone Record identifier Endocervical curettage alone (ECC) Patient demographics County of residence Other procedures performed State or territory of residence Description of other procedures performed Zip code of residence Cervical diagnostic procedures paid by NBCCEDP funds? Date of birth Cervical diagnosis Status of final diagnosis Race 1–5 Final diagnosis Hispanic or Latino origin Final diagnosis–other Cervical screening Previous Pap test Date of final diagnosis Date of previous Pap test Stage at diagnosis (legacy field) Indication for Pap test Cervical cancer treatment Status of treatment Cervical diagnostic referral date Date of treatment status Bethesda system used Additional breast imaging procedures Additional mammographic views Specimen adequacy of Pap test Ultrasound Specimen type for Pap test Film comparison to evaluate assessment incomplete Results of Pap test (Bethesda 1991) Final imaging outcome Results of Pap test (Bethesda 2001) Date of final imaging outcome Other Pap test result Breast diagnostic procedures Repeat breast examination/surgical consultation Date of Pap test Biopsy/lumpectomy Pap test paid by NBCCEDP funds Fine-needle/cyst aspiration Result of HPV test Other procedures performed Date of HPV test Description of other procedures performed HPV test paid by NBCCEDP funds Breast diagnostic procedures paid by NBCCEDP funds? Diagnostic work-up planned for cervical dysplasia or cancer? Breast diagnosis Status of final diagnosis Breast screening Breast symptoms Final diagnosis Clinical breast examination (CBE) result Date of final diagnosis/imaging Date of CBE Stage at diagnosis (legacy field) CBE paid by NBCCEDP funds? Tumor size (legacy field) Previous mammogram Breast cancer treatment Status of treatment Date of previous mammogram Date of treatment status Indication for initial mammogram Cancer case information from link-ages with central cancer registry Registry linkage status Breast diagnostic referral date Registry date of diagnosis Mammography test result Registry histologic type Date of mammogram Registry behavior Mammogram paid by NBCCEDP funds Registry summary stage Diagnostic work-up planned for breast cancer? Registry collaborative stage derived AJCC stage group Registry collaborative stage tumor size Registry collaborative stage extension Registry collaborative stage lymph nodes Registry collaborative stage metastasis at diagnosis Registry primary siteThe CDC developed a patient tracking and data management software application that is available at no cost to grantees called the Cancer Screening and Tracking System (CaST). The software application is used by grantees to manage local program data and data collection and to facilitate extraction of the MDE data set for submission to the CDC. About half of CDC grantees use CaST, whereas others have purchased their own data management software or have integrated the MDEs into a larger health data system in their state or territorial health department. The way in which grantees collect and manage their data varies widely. Some have web-based electronic systems that allow local providers to submit data directly, whereas others maintain paper-based systems in which providers submit completed forms on each woman for data entry and management. All grantees must conduct random data audits through medical chart reviews as part of the legislative requirements of the program to ensure data validity.
The CDC contracts with a data management firm that maintains the MDE data, provides technical support to NBCCEDP grantees, and provides data management and analytic support to the CDC. Functions carried out by the data management contractor include assistance with MDE data translation and reporting requirements, website and software support, and guidance for the development and implementation of data systems. To enhance data quality, the CDC and the data contractor have developed software tools, quality-assurance algorithms, and a data user’s manual detailing the MDE reporting requirements and submission process. These materials are routinely reviewed and updated in response to changes in clinical guidelines and reporting requirements. The CDC provides data-related training and technical assistance to grantees through site visits, webinars, and seminars at business meetings.
The CDC supports a continuous MDE submission and feedback process (Fig. 1), which begins with the grantees’ biannual submission of a standardized MDE file to the data contractor through a secure NBCCEDP resource website. Grantees are required to use an “MDE Edits” application to perform basic validation checks on the data before submission. The MDE Edits application allows grantees to produce reports that summarize the frequency of invalid (or missing) and inconsistent data within and across patient records. The CDC expects that less than 5% of records within a data file will contain a validation edit. To capture record updates that occur between submissions, the MDE data set consists of both new records and previously submitted records. With each MDE submission, grantees also are required to submit an accompanying narrative report describing any programmatic or data system changes that have a potential or known impact on data quality.
Figure 1.The Minimum Data Elements (MDE) submission process is illustrated. NBCCEDP indicates National Breast and Cervical Cancer Early Detection Program; CDC, Centers for Disease Control and Prevention.
Once the data are received by the data contractor, rigorous processes are in place to review and ensure completeness and data quality. The data contractor validates the record format, count and length, and performs validity checks for the completeness of single and related fields within and across records for unique patients. Files that do not meet data-validation standards are returned to grantees for investigation and correction. Once data issues are adequately addressed, the data contractors aggregate the data into a single SAS data-analysis file (SAS Institute Inc., Cary, NC). Finally, a set of standardized feedback and quality-assurance reports are produced for each grantee based on their data and for the national program as a whole.
CDC Use of the MDEsThe CDC uses the MDE data set for a variety of purposes. Below, we describe 3 main uses: 1) program monitoring and improvement, 2) evaluation and research, and 3) policy development.
Program monitoring and improvementOngoing monitoring of outcomes on the basis of MDE data allows grantees as well as the CDC to make adjustments designed to better manage the program, improve program effectiveness, and identify new goals as part of ongoing planning efforts. One valuable monitoring effort that supports NBCCEDP program improvement involves the feedback and quality-assurance reports that are produced for each MDE submission. These reports include: edits and frequencies to validate the data, program performance quality indicator measures to assess trends in data and service quality, graphic depictions of client characteristics and screening outcomes, summary reports comparing grantee-specific screening outcomes with national program screening outcomes, audit reports that identify specific records and clients requiring investigation and follow-up, and management reports for the CDC to compare measures across all grantees. One report includes a set of 11 NBCCEDP core indicators which allow the CDC to monitor grantees’ performance (Table 2). These indicators represent the highest priorities of the NBCCEDP. An example of a report based on 1 of these indicators (the percentage of women aged ≥50 years screened for breast cancer) is provided in Figure 2.
TABLE 2.National Breast and Cervical Cancer Early Detection Program Core Performance Indicators
Indicator Type Indicator Description CDC Standard, % Screening the NBCCEDP priority population Women newly enrolled for cervical screening have not had a Pap test in the past 5 y ≥20 Mammograms are provided to women aged ≥50 y ≥75 Complete and timely clinical follow-up to cervical cancer screening Women with abnormal screening results complete diagnostic follow-up ≥90 Diagnostic follow-up is completed within 90 d of the abnormal screen ≥75 Women diagnosed with cancer or high-grade precancerous lesions initiate treatment ≥90 Treatment for invasive cervical cancer is initiated within 60 d ≥80 Treatment for high-grade precancerous lesions is initiated within 90 d ≥80 Complete and timely clinical follow-up to breast cancer screening Women with abnormal screening tests complete diagnostic follow-up ≥90 Diagnostic follow-up is completed within 60 d of the abnormal screen ≥75 Women diagnosed with breast cancer initiate treatment ≥90 Treatment for breast cancer is initiated within 60 d ≥80 Figure 2.This is an example of a Minimum Data Elements (MDE) feedback report that measures grantee performance in meeting an indicator of National Breast and Cervical Cancer Early Detection Program (NBCCEDP) screening priorities: providing mammography screening to women aged ≥50 years. The table and histogram provide information on how Grantee X performed relative to the NBCCEDP target, the NBCCEDP aggregate, and other grantee programs. Min-Max indicates minimum-maximum.
After each data submission, the feedback and data-quality reports are reviewed during a conference call that includes grantees, their data contractor, and their CDC program consultant. Before the call, the data contractor reviews the reports and prepares a set of discussion points, which are shared with the CDC and the grantee and used to guide the discussion. Discussion topics may include both data-quality issues (eg, missing data) and service-delivery issues (eg, timeliness of diagnostic follow-up). After the call, the data contractor summarizes areas of concern as “action items” for the grantee to investigate. Grantees then provide a written response to each action item as part of a narrative summary that is included with their next data submission. The performance of grantees has improved since the introduction of the data feedback and quality-assurance reports along with other performance management strategies.11
Research and evaluationThe MDE data set represents 1 of the largest single breast and cervical cancer screening databases and, thus, offers opportunities for research and evaluation. CDC researchers and epidemiologists use MDE data to study the screening practices of grantees and evaluate the extent to which funded programs achieve the objectives of the authorizing legislation. Below, we provide examples of MDE analyses conducted by CDC researchers and others.
An analysis using a subset of core program performance indicators to assess service-delivery benchmarks for breast cancer screening in the NBCCEDP concluded that women who were screened by the NBCCEDP received diagnostic follow-up and initiated treatment within the pre-established program guidelines.12 In another study based on MDE data, an assessment of the use of colposcopy and repeat Pap testing as follow-up testing for women with abnormal cervical cancer screening results, researchers observed that follow-up colposcopic biopsies were performed more frequently on women aged <30 years compared with those who were older. These results suggested that providers may be overly aggressive with this population and that interventions may be needed to encourage more appropriate follow-up tests for women aged <30 years.13 Another study based on MDE data assessed whether the percentage of NBCCEDP patients with low-grade squamous intraepithelial lesions who underwent follow-up colposcopy changed after the 2002 publication of the 2001 Consensus Guidelines for the Management of Women with Cervical Cytological Abnormalities,14 which recommended immediate colposcopy. Results demonstrated that the percentage of women who underwent colposcopy increased by 9% between 2000 to 2002 (preguidelines) and 2003 to 2005 (postguidelines), consistent with the guideline.15
From 2004 to 2006, the CDC conducted a data validation study to verify the quality of the MDEs, assess their usefulness for program management and evaluation, and identify potential issues that may affect data validity. The overall conclusion from the comprehensive validation project was that, with few exceptions, the MDEs were valid and consistent with sociodemographic and clinical data from the medical records.16 These research and evaluation studies provide evidence that NBCCEDP patients do receive appropriate and timely clinical services, a requirement of the law establishing the program, and that the NBCCEDP employs quality data systems.
Policy development and analysisMDEs can also be used to educate policymakers who are developing federal or state legislation or policy. For example, the original law establishing the NBCCEDP (Public Law 101–354) appropriated significant federal resources for the early detection of breast and cervical cancers but explicitly prohibited the use of funds for treatment. Beginning in the late 1990s, efforts by the CDC, advocacy organizations, and legislators began in earnest to address this policy gap to argue that the role of the NBCCEDP should be expanded to include treatment. At the CDC, researchers were convened to review MDE data and determine what additional research would be needed to better understand and document the challenges grantees faced in securing treatment resources. In response, the Breast and Cervical Cancer Prevention and Treatment Act (BCCPTA) of 2000 (Public Law 106–354) was passed to address the treatment gap.17,18 MDE data were used later to analyze the impact of the BCCPTA.19
A Federal Advisory Committee was chartered on September 12, 1994, as required by the legislation authorizing the NBCCEDP. The committee meets annually to provide guidance on relevant clinical and programmatic issues facing the NBCCEDP. MDE data typically are included in updates that the CDC provides to the committee.
Examples of 2 Grantees’ Use of MDE DataTwo examples of MDE data use by grantees are provided below. The Louisiana program example reflects their use of data for monitoring and evaluation. In the second case, Washington State illustrates the use of their data system for program planning and management. In both examples, program staff used program data to inform program improvement.
LouisianaThe Louisiana State University Health Sciences Center School of Public Health, a NBCCEDP grantee, administers the Louisiana Breast and Cervical Health Program (LBCHP). Below, we describe how the LBCHP used program data to estimate mammography rescreening rates among women screened through the LBCHP and to monitor the timeliness of diagnostic follow-up for women with abnormal Pap test results. In both cases, LBCHP staff used Tableau Software (Seattle, Wash), a graphic visualization software package that was designed to help interpret and graphically depict data for communication with stakeholders.
Rescreening mammographyThe LBCHP data team, as part of an evaluation of the early detection of breast cancer among its program participants, calculated the mammography rescreening rate for women who initially were screened from October 2005 through March 2007 (n53776). The LBCHP network of providers recommends annual mammography screening, but biennial screening was still considered on time. For this evaluation, women were assessed over a 30-month period. To adjust for early (<9 months) and late (>12 months) appointments, the team used an interval of 9 to 27 months postindex screening mammogram and observed that only 32% of women had been rescreened. Further analyses revealed that, among women who had not returned by 16 months postindex mammogram screening, women ages 56 to 81 years had an even lower rescreening rate than women ages 50 to 55 years. It was concluded that, in the LBCHP, older women, who are at higher risk for breast cancer, were less likely to return for rescreening annually or biannually.
To illustrate these study results to providers, the team created graphic depictions of the findings using Tableau Software (Fig. 3). LBCHP staff then collaborated with providers to begin a text messaging intervention to improve rescreening rates. For this part of the intervention, women who were due for rescreening were sent a cell phone text message at 17 and 21 months after their index mammography in an effort encouraging them to be rescreened.
Figure 3.The percentage of Louisiana Breast and Cervical Health Program clients ages 50 to 57 years who received rescreening mammograms is illustrated according to age group.
Timeliness of cervical cancer screeningsTo monitor program performance and to implement provider assessment and feedback, the LBCHP uses Tableau Software to graph provider cervical cancer screening performance by site as measured by 3 performance indicators developed by the CDC: 1) provision of diagnostic follow-up within 60 days of patients’ abnormal Pap test results; 2) initiation of cancer treatment within 60 days of patients being diagnosed with invasive carcinoma; and 3) initiation of cancer treatment within 90 days for patients diagnosed with moderate to severe cervical dysplasia. The LBCHP also produces standardized monitoring data reports. These progress reports enable providers to assess specific core indictors while demonstrating whether the provider is meeting LBCHP goals compared with state and national levels.
For example, Figure 4 demonstrates that Provider E failed to meet the performance indicator for cervical cancer screening and diagnosis. Consequently, LBCHP staff were able to present the provider with visual confirmation that their site’s performance was disproportionately out of range. LBCHP staff and the provider identified problems with timely follow-up and instituted improvements to meet the performance standards of the LBCHP. Well designed graphic illustrations helped to facilitate discussion between the data team and providers about needed quality improvement.
Figure 4.A comparison of the time between cervical screening and cancer diagnosis and between cancer diagnosis and the initiation of treatment (Tx) is illustrated among patients at 5 Louisiana Breast and Cervical Health Program service providers.
Washington StateThe Washington State Department of Health, as a grantee of the NBCCEDP, uses its funds to administer the Breast, Cervical, and Colon Health Program (BCCHP), which subcontracts with 7 regional organizations throughout the state, known as prime contractors, to operate the BCCHP within their regions by working with a range of local providers. The BCCHP service-delivery network consists of more than 600 subcontracted providers, including mammography facilities separate from private clinic offices and laboratories that provide surgical pathology and cytology services.
To manage data in its decentralized service delivery network, the BCCHP uses Med-IT, an online medical information tracking system developed by OxBow Data Management Systems, LLC (Bellevue, Neb). Through Med-IT, BCCHP has a complete web-based health information system that accommodates its service delivery network. Med-IT allows BCCHP prime contractors to directly input patient information, automatically compute program eligibility, report MDEs, track and provide follow-up services as necessary for patients screened, produce provider-specific data reports that support quality-assurance and program-improvement efforts, and generate reimbursement authorization for their providers.
Prime contractors use Med-IT to monitor service delivery and reimburse providers; they also use the system to report their data to the BCCHP. BCCHP staff and prime contractors are able to review data in both “real time” and retrospectively. Like the Louisiana LBCHP, Washington produces standardized reports, including performance assessments based on NBCCEDP indicators to monitor the program. These reports may address the performance of a prime contractor, the performance of an individual provider, or the treatment received by an individual patient. Prime contractors use these data reports to identify both successes and problems in their provision of services and training-assessment needs.
The BCCHP also uses the data for program management purposes. Each year, the BCCHP projects the number of screenings that can be conducted in the coming year given the anticipated budget. The number of women screened fluctuates slightly from year to year based on several factors, including overall program resources and the number of women who require additional diagnostic procedures because of abnormal screening results. By using a clinical estimate tool in Med-IT, prime contractors are able to project the number of women who can be screened and the overall clinical costs for the coming program year. The tool uses historical data to account for projected differences between rates of abnormal screening results among women who are screened for the first time and rates among those who have been screened previously.
Throughout the year, BCCHP prime contractors monitor data on the number of women screened and the cost of their screening and adjust client recruitment as indicated to meet target screening goals and stay within their budgets. Figure 5 illustrates a comparison of enrollment screening for the first quarter of the 2008 and 2009 program years for 1 BCCHP prime contractor.
Figure 5.A comparison of enrollment screening is illustrated for the first quarter of the 2008 and 2009 program years (PY) for 1 Washington State Breast, Cervical, and Colon Health Program prime contractor. The bars represent "September, August, July" left to right.
ConclusionThe NBCCEDP, administered by the CDC, has been integral in screening millions of medically underserved women for breast and cervical cancer since its inception in1991. Data collected by NBCCEDP grantees have been used by the CDC and grantees for diverse purposes, including program monitoring and improvement, evaluation and research, and policy development and analysis. Most important, data collected on the women served through the program enable the CDC to ensure that NBCCEDP patients receive quality screening and diagnostic services in a timely fashion. The NBCCEDP data system described in this article may be a useful model for other clinical service delivery programs.
AcknowledgmentsWe thank Jennifer Hayden from Louisiana State University’s Health Sciences Center, New Orleans School of Public Health, Sections of Epidemiology and Health Promotion and Disease Prevention; and Pama Joyner from the Office of Community Wellness and Prevention in the Division of Community and Family Health, Washington Department of Health, for their contributions to this article.
The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. The use of trade names and commercial sources is for identification only and does not suggest endorsement by the US Department of Health and Human Services.
FUNDING SUPPORT
This Supplement edition of Cancer has been sponsored by the US Centers for Disease Control and Prevention (CDC), an Agency of the Department of Health and Human Services, under the Contract #200-2012-M-52408 00002.
FootnotesThis article has been contributed to by US Government employees and their work is in the public domain in the USA.
CONFLICT OF INTEREST DISCLOSURES
The authors made no disclosures.
REFERENCESRetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.3
