February 3, 1999 Meeting Summary:
Table of Contents
I. Record of Attendance
II. Welcome and Introductory Information
III. Presentations and Committee Discussion
IV. Concluding Remarks
VI. The Addenda
Record of Attendance
||Ex Officio Members
|Dr. Morton Schwartz, Chair
||Dr. Steven Gutman, FDA
||Ms. Kay Setzer (HIMA)
|Dr. David Baines
||Dr. Robert Martin, CDC
|Dr. Regina Benjamin
||Ms. Judith Yost, HCFA
|Dr. Mary Burritt
|Dr. Ronald Cada
|Dr. Patricia Charache
||Dr. Edward Baker
|Dr. Susanne Gollin
|Dr. Verlin Janzen
|Ms. Diana Mass
|Ms. Deborah McHugh
|Dr. Toby Merlin
|Ms. Sharon Radford
|Dr. Patricia Saigo
|Mr. Elliott Segal
|Dr. Ulder Tillman
Centers for Disease Control and Prevention
Ms. Nancy Anderson
|Dr. Thomas Hearn
||Dr. John Ridderhof
Ms. Carol Bigelow
|Dr. Ed Holmes
||Ms. Renee Ross
Dr. Joe Boone
|Dr. Richard Keenlyside
||Ms. Rhonda Whalen
Ms. Carol Cook
|Mr. Kevin Malone
Ms. Maribeth Gagnon
|Dr. Adam Manasterski
Clinical Laboratory Improvement Advisory Committee
The Secretary of Health and Human Services is authorized under Section 353
of the Public Health Service Act, as amended, to establish standards to assure consistent,
accurate, and reliable test results by all clinical laboratories in the United States. The
Secretary is authorized under Section 222 to establish advisory committees.
The Clinical Laboratory Improvement Advisory Committee (CLIAC) was
chartered in February 1992 to provide scientific and technical advice and guidance to the
Secretary and the Assistant Secretary for Health regarding the need for, and the nature
of, revisions to the standards under which clinical laboratories are regulated; the impact
on medical and laboratory practice of proposed revisions to the standards; and the
modification of the standards to accommodate technological advances.
The Committee consists of 20 members, including the Chair. Members are
selected by the Secretary from authorities knowledgeable in the fields of microbiology,
immunology, chemistry, hematology, pathology, and representatives of medical technology,
public health, clinical practice, and consumers. In addition, CLIAC includes three ex
officio members, or designees: the Director, Centers for Disease Control and Prevention;
the Commissioner, Food and Drug Administration; the Administrator, Health Care Financing
Administration; and such additional officers of the U.S. Government that the Secretary
deems are necessary for the Committee to effectively carry out its functions. CLIAC will
also include a non-voting liaison representative who is a member of the Health Industry
Manufacturers Association and such other non-voting liaison representatives that the
Secretary deems are necessary for the Committee to effectively carry out its functions.
Due to the diversity of its membership, CLIAC is at times divided in the
guidance and advice it offers to the Secretary. Even when all CLIAC members agree on a
specific recommendation, the Secretary may not follow their advice due to other overriding
concerns. Thus, while some of the actions recommended by CLIAC may eventually result in
changes to the law, the reader should not infer that all of the advisory committee's
recommendations will be automatically accepted and acted upon by the Secretary.
WELCOME AND INTRODUCTORY INFORMATION
Prior to calling the meeting to order, CLIAC Chair Dr. Morton Schwartz
acknowledged the death of Dr. Lemuel Bowie, a member of CLIAC, on December 25, 1998. Dr.
Schwartz recognized Dr. Bowie for his excellent work in the field of laboratory medicine
and expressed appreciation for his contributions to this Committee. Dr. Schwartz then
called the meeting to order. Dr. Edward Baker, Director, Public Health Practice Program
Office (PHPPO), welcomed the CLIAC members and noted that this was to be the last meeting
chaired by Dr. Schwartz. He expressed gratitude to Dr. Schwartz for his years of dedicated
service to the Committee, and for his wisdom, advice and leadership in dealing with many
complicated issues pertaining to the Clinical Laboratory Improvement Amendments of 1988
(CLIA) regulations. Dr. Baker then announced that Dr. Toby Merlin will assume the role of
CLIAC Chair at the next meeting. He also introduced Dr. Robert Martin, the new Director of
the Division of Laboratory Systems (DLS), and Dr. Devery Howerton, who will be the new
Branch Chief for the Laboratory Practice Standards Branch (LPSB). Following Dr. Baker, the
CLIAC members made self-introductions and disclosure statements of their relevant
financial interests as they relate to the topics to be discussed during the CLIAC meeting.
PRESENTATIONS AND COMMITTEE DISCUSSION
Clinical Laboratory Improvement Amendments of 1988 (CLIA) Update
Centers for Disease Control and Prevention (CDC) -
Dr. Thomas Hearn, Branch Chief, Laboratory Practice Assessment Branch
(LPAB), DLS, PHPPO, updated the Committee on two of CDC's areas of responsibility relevant
to CLIA. His first report was on the status of the possible transfer of the CLIA test
categorization and waiver activities from the CDC to the Food and Drug Administration
(FDA). Dr. Hearn noted that the transfer is being addressed by the CDC, FDA, and Health
Care Financing Administration (HCFA), and that the process for the transfer is still being
worked out by the agencies involved. He then briefed the CLIAC on several issues that
relate to genetic testing, including the recent CLIAC recommendations to add specific
requirements for genetic testing to the CLIA regulations. These recommendations are
currently being evaluated by the Department of Health and Human Services for incorporation
into the regulations where appropriate. The CDC is also involved in two research
activities to gather information on current laboratory practices in molecular genetic
testing. One of these is a project to identify and characterize the focus of quality
assurance and proficiency testing programs for molecular genetic testing, in an effort to
identify a test or group of tests that could be used to monitor quality in molecular
genetic testing. The other is a survey conducted by Dr. Margaret McGovern, Mount Sinai
School of Medicine, of quality assurance practices in biochemical genetic testing
laboratories. Both of these projects will provide information that will assist in
determining appropriate quality standards for genetic testing laboratories.
Ms. Carol Cook, Health Scientist, LPSB, DLS, PHPPO, reported on CDC's
publication of the proposed model certification program for embryo laboratories, developed
as part of the implementation of the Fertility Clinic Success Rate and Certification Act
of 1992 (FCSRCA). The proposed model program was published as a notice with opportunity
for comment in the Federal Register on November 6, 1998. After analyzing the 15
comment letters received in response to the notice, the CDC will revise the model as
appropriate, and publish the final model certification program in the Federal Register.
The finalized version will be distributed to State officials as described in the FCSRCA.
Health Care Financing Administration (HCFA) -
Ms. Judy Yost, Director, Division of Outcomes and Improvements (DOI),
Center of Medicaid and State Operations (CMSO), HCFA, summarized HCFA's current CLIA
implementation activities. She referred the Committee to the statistical report containing
data on laboratory certification, CLIA-exempt states, accreditation organizations, survey
deficiencies, and enforcement. She stated that HCFA and CDC are working on the final
regulation for quality control (QC), and that development of this regulation is being
given the highest priority . She added that the framework for the QC regulation is based
on previous recommendations made by the CLIAC. Ms. Yost also mentioned that the CLIAC
recommendations concerning genetic testing are being considered for incorporation in the
CLIA regulations. Ms. Yost then reported that HCFA would meet the March 31, 1999, deadline
to have their computer systems certified for AY2K@ and that HCFA and CDC have sent out
letters to manufacturers and proficiency testing providers, urging them to ensure that
their systems will be operational in 2000. Last, Ms. Yost discussed program
integrity/fraud and abuse investigations being conducted by HCFA in coordination with
other government agencies, including the Department of Justice. She emphasized, as in
previous CLIAC meetings, that these investigations are not conducted as part of, or in
conjunction with, CLIA inspections.
Following Ms. Yost's presentation, several CLIAC members had questions
about the HCFA statistical data. Dr. Baker noted the decline in survey deficiencies over
time, an indicator of improvement in laboratory quality. He posed the question of what is
the ultimate goal in the rate of these deficiencies? Ms. Yost said that although the goal
is to have 100% of laboratories in compliance with CLIA (and thus 0% deficiencies), it is
an incremental process with many factors that affect the data. Several members commented
on the fact that the statistics differ geographically, and two Committee members stated
that inspection statistics may not always be a true reflection of quality assurance
Food and Drug Administration (FDA) - Addendum
Dr. Steven Gutman, Director, Division of Clinical Laboratory Devices,
Office of Device Evaluation, Center for Devices and Radiological Health, FDA, reported on
new initiatives that are affecting the FDA process for review and classification of
clinical laboratory devices and reagents. He mentioned the FDA's regulation on analyte
specific reagents, which includes requirements for in-house (home brew) laboratory tests,
and gave the website for the Health Industry Manufacturers Association (www.himanet.com)
as a source of information on these regulations . He described the FDA's compliance policy
guides, which explain the three ways in which products may be marketed - for research,
investigational, or clinical use. Next, Dr. Gutman outlined changes in the 510(k) and
premarket approval processes, intended to make them more streamlined and user friendly.
Last, he discussed the Hazard Analysis Critical Control Points program, being piloted for
medical devices. In this program focused on preventing problems, manufacturers of medical
devices identify critical control points and their limits in the manufacturing process,
and assure that they are in control. If a device failure or problem is detected,
corrective action can be taken immediately to prevent further problems from occurring.
A CLIAC member asked whether there will be significant changes in the CLIA
test categorization process when it is transferred to the FDA. Dr. Gutman responded that
the transfer will be primarily administrative with no major changes intended. He added
that efforts are being taken to maintain consistency with the process currently in place,
including use of the same scoring system. He did note that there will be products to be
categorized for CLIA that are currently exempt from FDA review.
HCFA Inspections of Cytology Laboratories
- Addendum D
Ms. Cheryl Wiseman, Health Insurance Specialist, DOI, CMSO, HCFA ,
presented information on HCFA's specialized reviews of cytology laboratories. HCFA has a
contract to conduct on-site surveys of cytology laboratories for complaint investigation,
by recommendation of the regional office or state survey agency, or random selection. HCFA
determines which laboratories are to be inspected and directs the contractor to conduct
the surveys. The survey team consists of a team leader, cytotechnologist screeners, and a
board-certified pathologist with extensive training and expertise in cytopathology. The
surveys are outcome-oriented and take an educational approach in assessing the
laboratory's compliance with the CLIA requirements. As seen in the data Ms. Wiseman
presented, the percentage of laboratories with condition level deficiencies has decreased
over time, an indicator of improved laboratory performance.
Several CLIAC members asked for clarification of the statistical data, and
asked what HCFA's goals are in conducting these surveys. Ms. Wiseman explained that they
are trying to survey cytology laboratories in different geographic areas and laboratories
that routinely screen different test volumes. Ms. Yost also pointed out that in most
cases, HCFA inspectors do not have specialized expertise in cytology, and the contractor
surveys are focused on cytology practices and reporting, including a review of previously
reported cases. The Committee also discussed the potential value of conducting outcomes
research to determine whether implementation of CLIA has resulted in an increase in the
early detection of abnormal Papanicolaou (Pap) smears, and a decrease in invasive cervical
Update on Research on CLIA Cytology Requirements
- Addendum E
Ms. Rhonda Whalen, Senior Health Scientist, LPSB, DLS, PHPPO, gave a
brief chronology of cytology proficiency testing (PT) with respect to CLIA. She summarized
the applicable regulations and the steps in implementing the regulations, including
research projects that have been conducted in an effort to develop a cytology proficiency
testing program that can be administered on a national level.
Dr. Richard Keenlyside, Medical Officer, LPAB, DLS, PHPPO, discussed the
results of a CDC study (conducted under contract by Analytical Sciences, Inc.) that compared cytology screeners work performance to their scores on a
CDC glass slide proficiency test and a computer proficiency test developed by the CDC. In
this study, the work performance of 85 cytology screeners was measured using a pair of
cytoevaluators to retrospectively rescreen 500 Pap smears evaluated by each cytology
screener. The cytology screeners were proficiency tested on-site using a set of 10 glass
slides and in Atlanta using a computer test consisting of 10 challenges. The scoring
system, which consisted of the four diagnostic categories outlined in the CLIA
regulations, was used to evaluate cytology screener work performance and to score each
proficiency test. Data from the study demonstrated that there is a low, but definite
(unlikely to be a chance finding), correlation between an individual's work performance
and the results of both glass slide and computer proficiency tests. Dr. Keenlyside noted
that measurement uncertainties exist in a single 10 challenge proficiency test that need
further investigation. Information from cytology PT should, therefore, be used carefully
in conjunction with other performance measures as part of a laboratory's quality assurance
Public Comments - Addendum F
Dr. Diane Davey, representing the American Society of Cytopathology,
commented on the cytology study conducted by the CDC. She raised the following concerns,
which were echoed in a second public comment by Dr. George Birdsong, a cytopathologist
from Grady Memorial Hospital in Atlanta, Georgia.
There needs to be a higher level of correlation between PT and actual work
performance for cytologists to feel comfortable with computerized proficiency testing. CDC
must carefully consider the implications of failing cytology PT when it could mean the
potential loss of an individual's job.
The study data indicates many cytologists may be considered proficient by
one test and deficient by another test. Using the CLIA system for scoring the
computer-based PT, approximately 50% of individuals would fail. This does not represent
the true rate of problems in cytology. Either the computerized program, or the grading
system, or both, should be changed.
Data showing the correlation between glass slide PT and computer-based PT
were not presented, and the regulations currently specify glass slides.
Although slides were referenced prior to testing, they were not validated
by consensus of participants. There can be much variation in diagnostic slides.
Pathologists were not accurately represented in the study. Evaluation of
pathologists' locator skills is not a valid measure of pathologist performance, since most
pathologist do not screen slides.
The lack of pathologist review of discordant cases (disagreement between
cytologist and cytoevaluator) does not represent current practice where the pathologist is
responsible for the final determination of any abnormal reports. It is important to
correlate slide diagnosis with actual clinical follow-up.
Classification of HSIL cases as LSIL or ASCUS should not be considered a
The number of slides (500) evaluated may be too low in some cases. The
prevalence of disease in a population (or number of abnormals) should be used to determine
the number of slides needed to accurately measure work performance.
A few CLIAC members commented on the most clinically significant error
in reading Pap smears, which occurs when a high grade case is reported as normal, and
asked whether the correlations for this had been evaluated for glass slide versus
computer-based testing. They stressed that meaningful results in this study may be missed
by focusing on things that occur with higher frequency but are of lower significance. Dr.
Keenlyside agreed with these comments, but noted that since the numbers in this study were
small, it may not be possible to evaluate an even smaller subset of data. One Committee
member asked if there was a clinical utility in having the computer images trace a case
over time so that the review was not just a measure of performance on one point in time.
Another Committee member expressed several concerns with this study, some of which were
the same as those mentioned by Dr. Davey. Additional points included :
A 10 slide PT is not a valid measure of an individual's ability to read
Pap smears, especially in light of the fact that the percentage of abnormals in a
proficiency test is much higher (60%) than what is seen in the workplace (approximately
95% normal). When one fourth of the slides are abnormal it does not reflect the monotony
of the workday.
The skills and responsibilities of cytotechnologists and pathologists
differ, and it is not appropriate that they be tested in the same way.
There is a need to examine why there were discrepancies that were two
diagnostic categories apart. In the workplace, all abnormal slides are reviewed by a
pathologist. The cytoevaluators who reviewed discrepancies were not pathologists. The
study should have included follow up on the 43 slides called normal by cytologists but
determined to be HSIL by the cytoevaluators.
It is important to use validated slides and validated computer images. In
some cases the computer images may not reflect the microscopic image of certain cells.
Also, fewer glass slide PT sets should have been used and the test sets should not have
such a variable performance.
PT will identify individuals who have problems but continually testing
individuals who consistently pass PT is not necessary.
In conclusion, several CLIAC members stated that there are a number of
issues regarding cytology PT that must be considered. One member suggested that perhaps PT
should be used as a screening mechanism to determine which laboratories need closer
scrutiny or further review. One member stated that blind PT may never work in cytology and
that on-site surveys may be the best way to measure laboratory quality. Another member
stated that it might be helpful to have a presentation to CLIAC on the Maryland cytology
PT experience. Others emphasized that more information is needed on cytology PT.
HCFA Validation Inspections of Accredited
Laboratories - Addendum G
Ms. Sandra Farragut, Health Insurance Specialist, DOI, CMSO, HCFA,
gave an overview of HCFA's validation survey process, in which on-site inspections of a
sample of accredited laboratories are performed to determine if CLIA equivalency is being
sustained by the accreditation organization. She explained that the review teams used by
HCFA include both technical and nontechnical personnel in an effort to be objective. When
conducting the validation inspections, the team looks for serious, or Acondition level@
deficiencies, which are deficiencies that have the potential for directly affecting the
accuracy and reliability of test results. If HCFA finds a condition level deficiency that
was not identified by the accreditation organization, a disparity is noted, and disparity
rates are determined for each accreditation organization.
Ms. Farragut presented data from the FY >97 validation inspections and
said that the surveys for FY >98 should be completed soon. Regarding the FY >97
data, she stated that of 14,000 accredited laboratories, approximately 50% are accredited
by the Commission on Office Laboratory Accreditation (COLA), and 25% each are accredited
by the College of American Pathologists and the Joint Commission on Accreditation of
Health Care Organizations. Among the American Association of Blood Banks, the American
Osteopathic Association, and the American Society for Histocompatibility and
Immunogenetics, approximately 400 laboratories are accredited. Ms. Farragut also mentioned
that at this time, HCFA is revising the validation survey process in an effort to improve
communication with the accreditation organizations and decrease the disparity rate.
Public Comments - Addendum H
Dr. Stephen Kroger, Chief Executive Officer, COLA, commented on Ms.
Farragut's presentation. He commended HCFA for their approach to these inspections, and
gave a brief summary of COLA's analysis of the increase in the disparity rate for their
organization from FY >96 to FY >97. He explained the steps COLA took to investigate
this increase, and actions taken in response to their findings. He concluded with several
recommendations to improve the current validation survey process.
The Committee asked for clarification of several points with respect
to validation surveys. Ms. Yost referenced them to the CLIA regulations, where the process
is specifically described. She stated that in performing these inspections, HCFA is mainly
looking for trends, and acknowledged that there will always be some inherent differences
between a validation survey and the inspection performed by the accreditation
organization. She added that HCFA's plan to implement a process for conducting surveys
simultaneously by both groups may result in differences being resolved immediately.
Simultaneous surveys would also be a more streamlined process for laboratories being
inspected. CLIAC members asked whether validation surveys are performed in CLIA-exempt
states, and whether joint surveys are an option for states. Ms. Farragut reported that
results of validation surveys show that states are doing about the same or slightly better
than the accreditation organizations. She also said that simultaneous surveys may be an
option in the exempt states. Several CLIAC members noted that there are states conducting
surveys for HCFA that have higher deficiency rates and suggested that it may provide
useful information to send surveyors from a region or state with laboratories identified
as having many disparities to another area not known for having problems, and have them
conduct inspections in that region or state. This could illustrate whether the disparities
are true problems or a result of inconsistencies in the inspection process.
As Dr. Schwartz prepared to adjourn his last CLIAC meeting, he noted some
of the things he had learned in his experience as Chair of this Committee. Among the many,
he mentioned the significance of a level playing field, the difficulties of providing
laboratory services in rural areas of this country, and that compromise is important. He
expressed his appreciation to the CDC , HCFA, and the FDA for providing support to the
CLIAC during the time he served as Chair, and adjourned the meeting.
I certify that this summary report of the February 3, 1998, meeting of the
Clinical Laboratory Improvement Advisory Committee is an accurate and correct
representation of the meeting.
/S/ Morton K. Schwartz, Ph.D.