Table of Contents
 

• Centers for Disease Control and Prevention (CDC)
• Centers for Medicare & Medicaid Services (CMS) CLIA Update
  • Overview of 2005 National Cytology PT Results

• Regulatory Framework for Assuring Quality of Cytology Screening
• Cancer of the Cervix, An Overview
• Gynecologic Cytology Laboratory Practice and Methodology
• Cytology Proficiency Testing and New Technologies in Cervical Cancer Screening
• American Society of Clinical Pathology
• The Maryland Cytology Proficiency Testing Program
• The New York State Department of Health Cytopathology Proficiency Testing Program: Lessons Learned and Recommendations
• Limitations of the Ten Slide Proficiency Test Model
• Cytology Proficiency Testing Workgroup Report
• Analysis of Data from the Maryland Cytology Proficiency Testing Program
• Committee Discussion: Cytology Proficiency Testing

Clinical Laboratory Improvement Advisory Committee
 

The Secretary of Health and Human Services is authorized under Section 353 of the Public Health Service Act, as amended, to establish standards to assure consistent, accurate, and reliable test results by all clinical laboratories in the United States. The Secretary is authorized under Section 222 to establish advisory committees.

The Clinical Laboratory Improvement Advisory Committee (CLIAC) was chartered in February 1992 to provide scientific and technical advice and guidance to the Secretary and the Assistant Secretary for Health regarding the need for, and the nature of, revisions to the standards under which clinical laboratories are regulated; the impact on medical and laboratory practice of proposed revisions to the standards; and the modification of the standards to accommodate technological advances.

The Committee consists of 20 members, including the Chair. Members are selected by the Secretary from authorities knowledgeable in the fields of microbiology, immunology, chemistry, hematology, pathology, and representatives of medical technology, public health, clinical practice, and consumers. In addition, CLIAC includes three ex officio members, or designees: the Director, Centers for Disease Control and Prevention; the Commissioner, Food and Drug Administration; the Administrator, Centers for Medicare & Medicaid Services; and such additional officers of the U.S. Government that the Secretary deems are necessary for the Committee to effectively carry out its functions. CLIAC also includes a non-voting liaison representative who is a member of AdvaMed and such other non-voting liaison representatives that the Secretary deems are necessary for the Committee to effectively carry out its functions. Due to the diversity of its membership, CLIAC is at times divided in the guidance and advice it offers to the Secretary. Even when all CLIAC members agree on a specific recommendation, the Secretary may not follow their advice due to other overriding concerns. Thus, while some of the actions recommended by CLIAC may eventually result in changes to the regulations, the reader should not infer that all of the Committee's recommendations will be automatically accepted and acted upon by the Secretary.
 

CALL TO ORDER - INTRODUCTIONS/FINANCIAL DISCLOSURES
 

AGENCY UPDATES AND COMMITTEE DISCUSSION

Ms. Judy Yost, Director, Division of Laboratory Services (DLS), Survey and Certification Group (SCG), CMS, updated CLIAC on the status of proposed legislation, Proficiency Testing Improvement Act of 2005 (H.R.4568), which has passed the House and is now in the Senate Health, Education, Labor, and Pensions (HELP) Committee. She stated CMS was asked to brief the HELP Committee on the cytology proficiency testing (PT) regulations, the background for these regulations, CMS' planned approach to regulatory changes, and the cytology PT data collected to date. Ms. Yost indicated the HELP Committee intends to hold the bill so their deliberations could include CLIAC's cytology PT recommendations.

Ms. Yost reviewed the regulatory process, including writing a preamble justifying the proposed changes to the 1992 regulations, drafting the standards, conducting an impact analysis, obtaining clearance from the agencies administering CLIA (CMS, CDC, and, if needed, the Food and Drug Administration [FDA]), and providing response time for public comment on the proposed rule. Ms. Yost expressed commitment from both CMS and CDC to expedite the rule-making process once CLIAC's recommendations for cytology PT regulatory revision are submitted to the Secretary, Department of Health and Human Services (HHS). She stated the Committee would receive a progress report at the September 2006 meeting.

Prior to presenting a brief overview of the preliminary 2005 national cytology PT results, Ms. Yost explained to the Committee that testing initiated late in one calendar year may carry over into the next year if retesting has not been completed, making data analysis for a specific test year difficult. She further advised the members that, while 11 laboratories had not enrolled in cytology PT by the end of 2005, all individuals subject to cytology proficiency testing for 2005 were enrolled and tested as of June 19, 2006. She summarized the 2005 data from the state of Maryland and the Midwest Institute for Medical Education (MIME) programs including the overall pass rate; a breakdown of pass rates for cytotechnologists, primary and secondary screening pathologists, and locum tenens; and failure rates over time. In closing, Ms. Yost encouraged use of the CMS web address (www.cms.hhs.gov/clia) for additional information and provided the following list of CMS-approved cytology PT programs for calendar year 2006:

• Maryland Cytology Proficiency Testing Program
• College of American Pathologists (CAP)
• American Society for Clinical Pathology (ASCP) (acquired the MIME program)

Committee Discussion

• A member stated while CAP discourages primary screening by pathologists, many pathologists might perform primary screening only rarely (as little as one day a year). While not defending the practice of primary screening by pathologists, this member described the required testing of these individuals as primary screeners as a double burden.
• An invited speaker commented that in April 2006 the American Society of Cytopathology (ASC) adopted a position statement recommending that cytology laboratories use cytotechnologists for primary screening of pap smears except in emergency circumstances. Addendum B

Dr. Thomas Hearn, Associate Director for Laboratory Systems, DLS (proposed), NCPDCID (proposed), CDC, thanked Dr. Diane Solomon for serving as Chair of the Cytology Proficiency Testing Workgroup and the Workgroup members for their efforts. He acknowledged the challenge confronted by the Workgroup and commended the group for successfully putting aside individual interests to fulfill its charge from CLIAC. After reviewing the relevant portions of the CLIA Law and regulations, Dr. Hearn reminded the CLIAC members that implementation of the regulations addressing quality assessment, personnel requirements, on-site inspection, and other regulatory compliance activities have already improved the quality of cytology testing.

In describing the chronology of events related to cytology PT and impediments to the implementation of 1992 cytology PT regulations, Dr. Hearn discussed the development of computer-based testing. This potential PT option was recommended by CLIAC in 1993 in response to the reported lack of a sufficient quantity of glass slides to implement a national cytology PT program. He summarized three CDC studies to evaluate the efficacy of using virtual slides to replace glass slides, which supported the use of computer-based testing in lieu of glass slide testing. Dr. Hearn explained to the Committee that to replace glass slide testing with computer-based testing would, however, require revisions to the 1992 CLIA regulations. In addition, he identified other variables that can affect cytology PT outcomes. Dr. Hearn concluded with a description of the collaborative nature of the rule-making process and reminded the members that CLIAC's focus should remain on recommendations to facilitate revisions to the 1992 CLIA cytology PT regulations.

*Note: The addendum was revised from material provided in the Committee's notebooks to reflect last minute updates by the presenter.
 

Herschel W. Lawson, M.D., Division of Cancer Prevention and Control, National Center for Chronic Disease Prevention and Health Promotion, CDC, presented information on the surveillance and incidence rates, screening, diagnosis, and associated costs of cervical cancer using data from the National Program of Cancer Registries and the Surveillance, Epidemiology, and End Results program. He pointed out that cervical cancer is the 11th most common cancer occurring in U.S. women, noting the most important factor contributing to cervical cancer is the absence or infrequency of receiving a screening test. In U.S. women, the incidence of cervical cancer has been reduced by 75% due to screening and early detection has increased survival rates. The estimated annual cost of screening and follow-up of abnormal Pap tests is more than $4 billion per year.

Committee Discussion

• A member asked for clarification of the data comparing the 2002 and 2006 new cervical cancer cases versus numbers of deaths. He stated that while 2006 appeared to have significantly fewer new cases, there did not appear to be a corresponding decline in the death rate. In response, Dr. Lawson explained the 2002 data represent actual numbers while the 2006 figures represent American Cancer Society estimates, which will probably be higher when the 2006 data become available.

^*Note: The addendum was revised from material provided in the Committee's notebooks to reflect last minute updates by the presenter.
 

Dr. James V. Little, Assistant Professor of Pathology, Emory University School of Medicine, and CDC consultant to DLS, gave an overview of pap smear testing and cytology laboratory practices. He detailed the personnel skills required for cytological examinations, cell classification terminology, and the role of pap smear testing in the detection of gynecologic abnormalities and malignancies. Dr. Little reminded the members PT is only one aspect of a cytopathology laboratory quality system that includes quality control of staining and processing, personnel competency evaluation, cytology-histology correlations, and random re-screening of normal smears, as well as educational processes. He stated that despite low sensitivity the pap test remains an effective means for cervical cancer detection. As such, it is most effective when women are screened at regular, repeated intervals and when laboratory practices are optimized. He ended his presentation by stating that the role of the pap test in cervical cancer prevention is evolving.

Committee Discussion

• Dr. Little was asked if data were available on the number of abnormal smears detected because of the requirement to re-screen 10 percent of normal smears. He explained data exist, but the incidence rate of abnormal pap smears is sufficiently low that a 10 percent re-screen is not the most effective tool for identifying missed cases.

George G. Birdsong, M.D., Emory University School of Medicine and Grady Health System, thanked TriPath Imaging and Cytyc Corporation for their assistance in developing his presentation addressing cytology PT in the context of new technologies in cervical cancer screening. He described the FDA-approved TriPath Focal Point Slide profiler and Cytyc ThinPrep Imaging System, as well as the non-approved TriPath Focal Point Guided Screening system; discussed how different technologies impact the cytology screening process; and reviewed the three current sources of liquid-based pap tests (Cytyc's ThinPrep, TriPath's SurePath, and MonoGen's MonoPrep). Dr. Birdsong also briefly touched on the Digene Hybrid Capture 2, a molecular DNA test for 13 high-risk types of human papillomavirus (HPV). He stated that FDA approval includes triaging patients diagnosed with atypical squamous cells of undetermined significance (ASC-US) and co-testing (use in conjunction with the pap smear) as routine screening of women 30 years of age or older. He concluded with a summary of issues associated with attempting to apply cytology PT to new technologies. Automated screening presents numerous challenges such as those arising from PT slide source variability, stain variability, and morphologic differences due to the various preparation techniques. The instrumentation, according to the manufacturers, is not ready to use for PT because it does not allow repeat testing of the same slide or reading a slide on multiple instruments.

Committee Discussion

• A member inquired about screening the same slide on the same type of instrument, but in different laboratories. Dr. Birdsong replied that he was unaware of any peer-reviewed literature on this subject.
• Dr. Turner asked if data were available on the percentage of conventional versus liquid-based preparations for pap testing being performed. Dr. Herschel Lawson stated the results of an independent CDC survey would be published in the next few months in Obstetrics and Gynecology showing that 80-85 percent of all cytology testing is currently performed using liquid-based specimens. A member commented that, while the conventional pap remains a very good screening test, the option to perform ancillary testing on liquid-based specimens has generated preference for this preparation among gynecologists.

Dr. Thomas Bonfiglio, PT Medical Director, ASCP, and Ms. Rhonda Metzler, PT Senior Manager, ASCP, presented an overview of the structure of ASCP's CMS-approved cytology PT program, summaries of test results, and proposals for regulatory changes. Dr. Bonfiglio described the program as a collaborative effort to improve laboratory quality and serve its members. Ms. Metzler detailed the ASCP program structure including the proctor system, slide validation standards, and evaluation of testing results. She also reviewed responses and rationales for test result appeals from participants. Dr. Bonfiglio presented ASCP's recommendations for changes to the regulations as follows: reduce the required testing interval from annual to every two or three years supplemented with educational requirements in alternate years, change the grading scheme to reflect current patient management practices, and eliminate the automatic failure feature. He urged HHS to implement these changes in 2006. Dr. Bonfiglio suggested that PT programs could also be improved by requiring field validation of slides. He concluded with an update of the activities of the new ASCP GYN PT & Assessment Committee.

*Note: The addendum was revised from material provided in the Committee's notebooks to reflect last minute updates by the presenter.

Committee Discussion

• A member asked how it would be possible to misclassify a high grade squamous intraepithelial lesion (HSIL) slide as negative in light of ASCP's stringent validation requirements for test slides classified as HSIL, i.e., five reviewers must agree on the interpretation and histology/cytology correlation must be documented. Dr. Bonfiglio stated cytology is subjective and intra-observability variation makes it possible to misclassify a slide, but ASCP hopes that by adding a requirement for 90 percent field validation by at least 40 individuals misclassification will not be an issue. He noted that earlier in the program, when slides were reviewed by three pathologists and field validation was performed by the examinees, a few individuals misread HSIL slides. Dr. Bonfiglio stressed the importance of increasing the number of slides per test set and removing the possibility of an individual failing based on misinterpretation of a single slide.
• Another member registered concern over the proposed changes to the scoring system that could result in failure to identify individuals interpreting pap smears incorrectly. Without an integrated educational process, such individuals could continue providing wrong diagnoses. The member suggested CLIAC recommend individuals identified as failing a testing event should participate in targeted educational processes and strive to achieve the highest attainable score. Another member concurred, but added that patient and clinician education is also needed, noting that 90 percent of cervical cancers occur in women who have either never been screened or have not been screened in the past five years, or whose pap smear is reported as abnormal but whose case is subsequently mismanaged; in contrast, interpretive errors account for 5-10 percent of missed cases.
• The same member expressed concern that the low number of PT appeals in 2005 could be attributed to the fact that the first individuals tested were unaware they could appeal their scores. Dr. Bonfiglio explained the MIME program (now ASCP) built checks into its PT program to address this issue. When an individual failed PT, the slide was pulled and subjected to a blinded review immediately. This de facto appeal process resulted in the PT program removing slides that had fallen out of validation and reclassifying individuals who failed based on such slides to passing status. Every slide in the program has now been validated at the 90% level with a minimum of 40 reviews.
• Dr. Turner requested and received clarification that individuals employed in more than one laboratory are required to undergo PT at only one site.
• One member noted the PT result data show more individuals choose to discontinue testing rather than attempt a second or third test and asked what is known about that group. Dr. Bonfiglio responded that the history of cytology PT demonstrates few examinees truly fail, while a greater number of individuals stop performing gynecologic cytology after failing the first test set and are no longer required to participate in the examination process. He further stated that when evaluating cytology PT effectiveness, it is important to consider not only the number of failures but also the number of individuals who have decided to cease PT and screening.

The state of Maryland submitted a written overview of the Maryland Cytology Proficiency Testing Program for CLIAC's review.
 

Ms. Deirdre Astin, Deputy Director of the Clinical Laboratory Evaluation Program at the New York State Department of Health, Wadsworth Center, began her presentation by providing the background and history of the New York State Department of Health Cytopathology Proficiency Testing Program. She stated the original state PT program was mandated in 1964, and an annual on-site glass slide evaluation program was established in 1968. Ms. Astin talked about the issues the program faced after changing to a biennial testing format in 1978 and then compared test design and grading systems of the New York State cytology proficiency program with those of CLIA. She compared performance statistics of the New York State program with the national statistics of the MIME program and concluded by discussing observations on test performance and considerations for CLIAC to include in their discussion on cytology proficiency testing.
 

Dr. George K. Nagy, Section Head, Pathology, New York State Department of Health, Wadsworth Center, began by stating the current ten-slide cytology proficiency testing model lacks the discriminatory power to differentiate between competent and incompetent test takers. He showed how the ten-slide test could misclassify test takers, with a high proportion of competent individuals failing and a high proportion of incompetent individuals passing. He went on to say that a competent examinee failing a test event (type 1 error) is the lesser problem because of the high probability that the individual would pass the second test event. However, an incompetent examinee passing the test event (type 2 error) is a more serious problem as this individual will continue to screen patient specimens until the next test cycle. Using statistical examples, Dr. Nagy demonstrated how larger sample sizes provide greater discrimination in probability distribution with more accurate estimations. Therefore, use of large samples in PT could substantially reduce misclassifications, especially those arising from type 2 errors. He proposed the main objectives of cytology PT should be the detection and re-education of incompetent individuals followed by retesting to assure competency. In summary, Dr. Nagy proposed a more accurate assessment of competency would be an initial test consisting of 40-60 slides administered upon entry of pathologists and cytotechnologists into practice. Additionally, he suggested the initial assessment be followed by PT at five- to ten-year intervals as is currently the practice for recertification by many professional boards since the ability to screen slides does not diminish over time.
 

Dr. Diane Solomon, Senior Investigator, Division of Cancer Prevention, National Cancer Institute, National Institutes of Health, DHHS, provided CLIAC with a summary of proceedings from the Cytology Proficiency Testing Workgroup meeting. She began by restating the Workgroup's charge: "Openly discuss issues, consider comments and develop potential framework and options for regulatory revisions that will be reported to CLIAC for developing recommendations to HHS for assisting in the development of a proposed rule." Dr. Solomon informed the members the list of issues presented for the Workgroup's consideration was derived from organizational comments and presented the Workgroup's comments and suggested options for regulatory revision for each of the following cytology PT issues:

• individual vs. laboratory
• new technology
• testing frequency
• number of challenges
• categories of challenges
• number of challenges per category
• grading scheme
• validation
• test site
• retesting
• confidentiality

Dr. Devery Howerton, Chief, Laboratory Practice Evaluation and Genomics Branch, and Acting Chief, Laboratory Practice Standards Branch, DLS (proposed), NCPDCID (proposed), CDC, prefaced her presentation by clarifying for CLIAC the cytology PT changes that could be achieved through regulatory revision versus those requiring statutory change. She explained her objective was to use raw data from the Maryland Cytology Proficiency Testing Program to present examples using the scoring models proposed by the Workgroup. This could provide a scientific basis for scoring changes that CLIAC might recommend. She then summarized the outcomes from analyses of 1998-2004 raw data from the Maryland program, first by evaluating performance according to the CLIA grading scheme and then by comparing four alternate grading schemes to CLIA. Next, she compared the results of a simulated 20-slide test with the 10-slide test, emphasizing this did not represent a true comparison study and acknowledging the results were thus inconclusive. Dr. Howerton also noted the Maryland data were inadequate for evaluating test frequency. With respect to frequency of testing, however, she commented that a retrospective analysis of the New York cytology PT program performance data showed no significant change in failure rate following a change from annual to biennial testing. Dr. Howerton concluded by again identifying the proposed cytology PT options provided by the Workgroup for discussion and acknowledged the individuals providing analysis of the Maryland raw data.

*Note: The addendum was revised from material provided in the Committee's notebooks to reflect last minute updates by the presenter.

Committee Discussion: Cytology Proficiency Testing

The CLIAC Chair introduced Dr. Solomon as technical moderator for the CLIAC discussions and referred the Committee to the list of the cytology PT issues to be considered. Several Committee members sought clarification on what specific elements of cytology PT could be changed by revision of the CLIA cytology PT regulations. Dr. Howerton reiterated changes in number and category of challenges and frequency of testing could be addressed by regulatory revision but PT of individuals is required by the CLIA statute and could not be addressed via this process.

Dr. Solomon reviewed the Workgroup's discussion and asked CLIAC to deliberate and make recommendations based on the Workgroup's options for regulatory revisions. The following represents a per-issue summary of options for regulatory revision identified by the Workgroup, the Committee's deliberations, and final CLIAC recommendations.

Note: Questions intended to stimulate CLIAC's consideration of each issue were provided as part of the Committee reference materials, but were not specifically used to guide the members' discussion. (Addendum M)

Individual vs. Laboratory

CLIAC members expressed concern that the statute required PT of individuals only in the subspecialty of cytology and stated this was unfair and should be an issue open for discussion. Several members also suggested CLIAC consider PT proposals presented in the PT providers' public comments. Dr. Turner reminded the Committee the objective of this meeting was to focus on the Workgroup report and develop recommendations addressing cytology PT requirements subject to change by regulatory revision. She reemphasized the issue of individual versus laboratory PT could not be addressed via this process. Before the Chair closed discussion of this topic, several members proposed the preamble language should reflect strong encouragement for laboratory participation in cytopathology educational programs in addition to individual proficiency testing. The Committee agreed, adding the CLIA interpretive guidelines could also be used to encourage oversight organizations/agencies and surveyors to determine if laboratories are participating in educational programs during on-site inspections. If non-participation is discovered then the site surveyor could identify available resources and encourage laboratories to enroll in cytopathology educational programs.

CLIAC Recommendations

• Use the preamble to encourage laboratories to participate in educational laboratory programs in addition to individual proficiency testing
• Oversight organizations/agencies and surveyors should determine if laboratories participate in educational programs and help to identify available resources if they are not

New Technology

One member suggested the regulatory language of "glass slide" be replaced with terminology that more realistically reflects the present and the future of gynecological testing. Another member cautioned the replacement language should not promote a specific technology nor inhibit the introduction of new technologies. The Committee agreed and recommended the regulatory language of "challenges" replace "slides" and challenges be defined as "case equivalent - glass slide, virtual slide, or other approved media." Several members also commented the use of "challenges" would allow flexibility to both PT providers in transitioning from using older technologies to newer technologies and to individuals in selecting a technology offering a greater comfort level. The CLIAC also recommended adding a regulatory requirement establishing a transition phase when new formats are introduced to allow individuals their choice of format for all retesting phases.

CLIAC Recommendations

• Change current language of "slides" to "challenges" to allow for the use of virtual slides or other media
• Define a challenge as a case equivalent - glass slide, virtual slide, or other approved media
• Add requirement for a transition phase for new technology (e.g., virtual slides), when the individual can request retesting with previous platform/format (e.g., glass slides)

Frequency of Testing and Number of Challenges

Dr. Solomon introduced the issues of test frequency and number of challenges, noting the two issues cannot be separated, since the Workgroup linked specific options for revisions to whether the number of challenges was increased concomitant with a decrease in frequency. The Workgroup favored a 20-challenge test every two-plus years. Dr. Solomon clarified the decreased frequency is justified since there is general agreement in the profession that individual skills do not deteriorate over time. In addition, she explained an increase in the number of challenges would decrease the effectiveness of gaming tactics. A member suggested the increased demands associated with preparing 20-challenge test sets should be offset in part by less frequent testing. Regarding the number of challenges and testing frequency, Dr. Solomon commented on Dr Nagy's proposal of 40- to 60-slide tests, saying the suggestion is theoretically appealing but testing twice with 20 slides within a six-year period would have the same mathematical effect as the larger test every six to ten years. Several Committee members commented that 40- to 60-slide tests could pose significant logistical challenges both for vendors and in terms of personnel coverage during time away from the bench. The Committee agreed to recommend 20 challenges for every test and to reduce the frequency of testing to a 3-year test cycle. PT providers noted initial testing of new personnel could still be accommodated as needed while routine testing in laboratories could be spread out over the three years.

CLIAC Recommendations

• Reduce the frequency of testing to a 3-year test cycle
• Use 20 challenges for every test (initial test and retests)

Categories of Challenges and Number of Challenges per Category

The Committee agreed with the Workgroup's proposed option of at least one challenge from each category based on a 20-challenge test.

CLIAC Recommendation

• Retain four response categories and continue to require at least one challenge from each of the four categories in each test

Grading Scheme

Prior to discussing specific workgroup comments and suggestions regarding the grading scheme, Dr. Solomon advised the Committee of the workgroup's approach to cytology PT grading issues. She explained their intent to propose a simplified, more valid grading system with response categories and scoring that reflect clinical practice decision points in patient management. Dr. Solomon further explained the workgroup's efforts to remain neutral in terms of affecting the current pass/fail rates throughout the process.

Dr. Solomon advised the Committee that the Workgroup strongly supported removing the automatic failure requirement (HSIL misinterpreted as negative resulting in a 15-point loss), explaining failure based on a single slide was deemed too punitive. In addition, the Workgroup supported eliminating partial credit for identifying a low grade squamous intraepithelial lesion (LSIL) as unsatisfactory (UNSAT). Dr. Solomon clarified the Workgroup's position that individuals should be able to distinguish between LSIL and UNSAT. She further noted elimination of partial credit would minimize examinees' ability to game the test system. However, misinterpretation of a "negative" as "unsatisfactory" should receive partial credit because in practice unsatisfactory slides result in the patient getting a repeat pap smear.

Dr. Solomon briefly reviewed cytological interpretation and clinical management models for the respective diagnoses of LSIL and HSIL, which differ from the standard of practice in 1992 when the current CLIA cytology PT regulations were finalized. She said the Workgroup was divided on whether to retain the requirement to distinguish between LSIL and HSIL.

Dr. Solomon told the Committee the Workgroup was also divided on whether there should be a unified or separate scoring system for pathologists and cytotechnologists and presented arguments supporting both approaches for consideration.

In favor of separate scoring systems:

• The cytotechnologist's primary responsibility is to alert the pathologist of a possible abnormality and refer cases, not to diagnose specific abnormalities.
• Responsibility for the final interpretation rests with the pathologist, not the cytotechnologist.

• Cytopathologists and cytotechnologists receive the same training and education in morphologic criteria, e.g., diagnosis of HSIL versus LSIL and normal versus unsatisfactory.
• Although the pathologist is ultimately responsible for interpretation of abnormals, in most cases the cytotechnologist's interpretation becomes the final diagnosis.

Committee Discussion

• Regarding the Workgroup's suggested removal of the automatic failure requirement, several members commented on the need to be sensitive to public perception when determining the scoring weight given to misinterpretation of HSIL as negative.
• In response to a member's question concerning how many examinees failed PT based solely on an automatic failure (misinterpreted an HSIL as negative and correctly interpreted the other nine test slides), Dr. Howerton responded automatic failure occurred with 11% of cytotechnologists and 7% of pathologists based on the Maryland program's data.
• A member noted that, for a 20-slide test, if the -5 was retained, an examinee misinterpreting one HSIL as negative would lose a total of 10 points and would have to provide completely correct responses to the remaining 19 slides in order to avoid a retest. Another member pointed out a 20-slide test might easily contain more than one HSIL slide, which could help discern whether an individual truly needs remediation in this area. A member moved to retain the -5 scoring for misinterpreting HSIL as negative, resulting in a significant penalty commensurate with the seriousness of the error but not in an automatic failure when using a 20-challenge test.
• Some committee members agreed that in many practices, pathologists "rubber stamp" the cytotechnologist's interpretation. In addition, if the cytotechnologist does not dot the cells, pathologists may not see the cells or re-screen the slides. A PT provider noted, in the previous year's testing, that pathologists provided the same response as their cytotechnologist screeners 90% of the time.
• A member emphasized the ultimate responsibility should fall on the decision maker, which in this case is the pathologist.
• Another member stated cytology PT holds examinees to a higher standard than clinical practice in that the latter allows the pathologist to include qualifying comments in the final interpretation.
• Dr. Hearn pointed out that 90% of the slides that are screened are interpreted as normal and never seen by a cytopathologist. Therefore, cytotechnologists have a high level of responsibility for reporting normal results and cytopathologists for reporting abnormal results. In principle, one could argue there is a balance in these responsibilities that might support a unified scoring scheme.
• Several Committee members commented that changes to PT should not additionally burden the cytotechnologists in comparison to current CLIA scoring, however the proposed unified system is slightly less rigorous for pathologists and slightly more challenging to cytotechnologists.
• A member proposed a unified scoring system with no penalty for distinguishing between LSIL and HSIL for pathologists.

Following extensive deliberations as summarized above, CLIAC agreed to recommend a unified scoring system that eliminates the automatic failure component yet imposes a significant penalty for misinterpretation of HSIL as negative. In addition, the recommended scoring system minimizes the distinction between LSIL and HSIL by requiring examinees to distinguish between these diagnoses but with no loss of points.

CLIAC Recommendation

• Change grading scheme to Unified Model X as shown below

	Model X-20-Slide Test - Unified
Correct Response	Examinee Response
	A - UNSAT	B-NEGATIVE	C - LSIL	D - HSIL
A - UNSAT	5	0	0	0
B - NEGATIVE	2.5	5	0	0
C - LSIL	0	0	5	5
D - HSIL	0	-5	5	5

Validation and Test Result Feedback

Dr. Solomon explained the reasoning behind the Workgroup's proposal to drop the requirement for biopsy confirmation of slides referenced as LSIL. While LSIL is a reproducible diagnosis, there are instances of cytologic LSIL that do not confirm colposcopically because colposcopy is not highly sensitive and these lesions are often transient and frequently regress in the interval between the time the pap smear is taken and the time of colposcopic biopsy. Therefore, the Workgroup supported dropping the requirement for biopsy confirmation of LSIL, thus making it easier for programs to collect examples of these cases, assuming appropriate slide validation.

The Committee discussed making field validation and disclosure of PT providers' validation process a regulatory requirement as follows:

• The issue of a slide gradually fading was raised as a factor in field validation. Additionally, PT providers noted liquid-based preparations tend to fade more rapidly than conventional smears.
• Specifically addressing the concern an HSIL slide may fall out of validation, one PT provider stated that every failure goes through a de facto appeal process, which includes technical evaluation, before results are reported.
• Another PT provider agreed that every failure for individuals is reviewed prior to results being returned to participants. Additionally, review of every slide when it is returned to the provider offers the opportunity to remove slides when the quality deteriorates. If an appeal is upheld or if a slide is removed after review, the grades are changed.
• Dr. Turner confirmed that when slides do not validate upon return the current PT providers adjust PT scores accordingly and notify affected examinees if their adjusted score results in a change to their pass/fail status.
• One PT provider responded that each time test sets are returned, the provider evaluates every slide as well as the participants' performance and applies algorithms to determine fair and equitable test set composition for the next round.

The Committee discussed recommending PT providers give examinees educational feedback of their test performance beyond the current requirement of informing examinees only of their area of test failure, noting such practice was already incorporated into cytology education-only programs.

• The Committee asked PT providers to comment on their ability to provide individual test scoring feedback without compromising test security. One provider cited no problems with test security because their slide set composition is determined for each testing event by application of computer algorithms to a large reservoir of slides. Another provider cited a potential security problem since slides in every test set were not always changed out before redistribution.
• A member suggested feedback be given only for discrepant answers.
• While stating it would be ideal to derive as much educational value as possible from each proficiency test, Dr. Solomon also commented on the additional burden to vendors if they are required to design cytology proficiency tests that function additionally as educational products, especially since participation in educational programs is already being encouraged.
• In response to a member's question, Ms. Yost reminded the Committee that CLIA prohibits examinee discussion or sharing of testing information prior to completion of the testing event.
• Several members suggested vendors furnish detailed feedback including examinee response, correct diagnosis, and any discrepancies. To prevent fraudulent sharing of results, the slides would not need to be specifically identified.
• Ms. Yost stated ideally cytology PT should correlate with clinical laboratory PT with respect to provider feedback and the required laboratory response. Regardless of whether a passing score is achieved, the laboratory is expected to review every incorrect response and to investigate the reason for any errors as a part of ongoing quality assessment.

CLIAC Recommendations

• Require biopsy confirmation of category D (HSIL/cancer) challenges, but not category C (LSIL) challenges
• Require field validation, monitor challenges continuously, and remove challenges that fail field validation
• Require validation procedures to be disclosed by the vendor
• Provide educational feedback for discrepancies

Test Site

Several members stated that laboratory directors should have the option to designate proctors. The Committee was informed that PT providers already determine the proctor requirements as well as retesting sites and was advised that specifying proctor requirements in the regulation could potentially hinder introduction of new technology or other proficiency testing formats.

CLIAC Recommendation

PT providers should continue to determine proctor requirements.

Retesting

Regarding the issue of retesting, the members agreed with the Workgroup's options for regulatory revision. (Addendum K)

CLIAC Recommendations

• Require PT providers to disclose the appeal process in writing
• Change language to state "individuals who score <90% must."(as opposed to "who fail")

Confidentiality

The Workgroup requested clarification from CMS regarding the issues of confidentiality of test scores.

Dr. Solomon asked Ms. Yost to describe how CMS addresses confidentiality issues. Ms. Yost informed CLIAC of an informational supplement on the CMS website (www.cms.hhs.gov/CLIA/downloads/Informational_Supplement.pdf) addressing frequently asked questions on cytology PT, including confidentiality issues and the release of individual test performance data. She encouraged CLIAC members to visit the CMS website to review the information provided. She reminded CLIAC the years 2005 and 2006 were educational cytology PT cycles therefore no individual scores are being reported to CMS. She described the CMS Cytology Personnel Records System (CYPERS), created in 2005, as a confidential records system linking laboratory demographics and cytology testing personnel. She indicated CMS would be addressing record retention issues in collaboration with CDC and agency lawyers this year. Additionally, she assured CLIAC that CMS protocol requires case-by-case evaluation for response to Freedom of Information Act (FOIA) requests for individual information and stated, in her experience, most litigation and subpoenas issued for release of information are directed to the individual, laboratory, and/or laboratory director and not to CMS.

Committee Discussion

• One member opened the discussion by asking for clarification of what individual PT performance information should be kept confidential. This member expressed concern that individuals failing all phases of remediation could go unrecognized. Ms. Yost responded all information in CYPERS is confidential and PT providers would not release scores without written consent from that individual.
• Several members expressed concern that individuals failing four rounds could be hired by another laboratory. Guidance to laboratory directors was recommended, advising the necessity to conduct primary source verification (i.e., obtain prior written authorization from the examinee to the PT provider for release of test results directly to the prospective employer) of prospective employees' PT status prior to hiring. Ms. Yost pointed out much of this concern should be alleviated by the fact that oversight organizations/agencies and CMS surveyors will have information, at least at the laboratory level, on individuals who have not passed.

CLIAC Recommendation

None

Note: CLIAC recommendations captured during the meeting are reflected in Addendum N.

Other Committee discussion

The Committee suggested reviewing and evaluating PT program performance for the next several years to decide if further scoring modifications are necessary.

A member asked when the PT regulation would be published and implemented. Ms. Yost told the Committee it is on the CMS regulatory agenda and reviewed the steps for moving a proposed regulation through the clearance and publication process.

Dr. Turner again acknowledged Dr. Solomon and the Cytology PT Workgroup and thanked the CLIAC members and partner agencies for their support and participation. The following reflects CLIAC's recommendations for revisions to the CLIA cytology PT regulations:

• Use the preamble to encourage laboratories to participate in educational laboratory programs in addition to individual proficiency testing
• Oversight organizations/agencies and surveyors should determine if laboratories participate in educational programs and help to identify available resources if they do not
• Change current language of "slides" to "challenges" to allow for the use of virtual slides or other media
• Define a challenge as a case equivalent - glass slide, virtual slide, or other approved media
• Add requirement for a transition phase for new technology (e.g., virtual slides), when the individual can request retesting with previous platform/format (e.g., glass slides)
• Reduce the frequency of testing to a 3-year test cycle
• Use 20 challenges for every test (initial and retest)
• Retain four response categories and continue to require at least one challenge from each of the four categories in each test set
• Change grading scheme to a new model that is the same for both technical supervisors and cytotechnologists
  
  

  	Model X-20-Slide Test - Unified
  Correct Response	Examinee Response
  	A - UNSAT	B-NEGATIVE	C - LSIL	D - HSIL
  A - UNSAT	5	0	0	0
  B - NEGATIVE	2.5	5	0	0
  C - LSIL	0	0	5	5
  D - HSIL	0	-5	5	5

  
   
• Require biopsy confirmation of category D (HSIL/cancer) challenges, but not category C (LSIL) challenges
• Require field validation, monitor challenges continuously, and remove challenges that fail field validation
• Require validation procedures to be disclosed by the vendor
• Provide educational feedback on discrepancies
• Require PT providers to disclose the appeal process in writing
• Change language to state "individuals who score <90% must."(as opposed to "who fail")

Dr. Turner announced the next CLIAC meeting is scheduled for September 20-21, 2006, and adjourned the Committee meeting.

I certify this summary report of the June 20-21, 2006, meeting of the Clinical Laboratory Improvement Advisory Committee is an accurate and correct representation of the meeting.

Lou Flippin Turner, Dr.P.H., CLIAC Chair

Dated: 9/7/2006