Clinial Laboratory Improvement Advisory Committee
(CLIAC)
Summary Report
May 10-11, 1995
U.S. DEPARTMENT OF HEALTH & HUMAN SERVICES
Public Health Service
Clinical Laboratory Improvement Advisory Committee
May 10-11, 1995
Summary
Table of Contents
I. Record of Attendance
II. Introduction
Clinical Laboratory Improvement Advisory Committee
Welcome and Announcements
Review of Conflict of Interest Forms and Disclosure Format
III. Topics
CLIA Update/CDC
CLIA Regulations/Archer Bill
Review of the CDC Process for Waiving Tests
Summary of Research and Data Analysis Activities
CLIA Update/HCFA Survey Data
Proposal for a Performance-Based Survey Process
Review of the Implementation of Personnel Standards
CLIA Requirements for Quality Control
V. Public Comments
VI. Summary
VII. Concluding Remarks
VIII. The Addenda
Record of Attendance
The Clinical Laboratory Improvement Advisory Committee (CLIAC) met at the
Centers for Disease Control and Prevention (CDC), Auditorium A, in Atlanta,
Georgia, on May 10-11, 1995. Those in attendance are listed below:
Committee Members Ex Officio Members
Dr. J. Scott Abercrombie Dr. Carlyn Collins, CDC
Dr. Paul Bachner Dr. Steve Gutman, FDA
Dr. Regina Benjamin Ms. Judith Yost, HCFA
Ms. Michelle Best
Ms. Virginia Charles
Dr. Susanne Gollin
Dr. Stanley Inhorn Executive Secretary
Dr. Verlin Janzen Dr. Edward Baker
Ms. Sandra Johnson
Dr. J. Stephen Kroger
Dr. Bereneice Madison
Dr. Kenneth Matthews Liaison Representatives
Dr. Wendell O'Neal Dr. Fred Lasky (HIMA)
Dr. Glenda Price
Ms. Deborah Reed
Dr. Patricia Saigo
Dr. Morton Schwartz
Mr. Elliott Segal
Dr. Ulder Tillman
Centers for Disease Control and Prevention
Ms. Nancy Anderson
Ms. Rosemary Bakes-Martin
Ms. Louise Barden
Ms. Carol Bigelow
Dr. Joe Boone
Ms. Sheila Boring
Ms. Genoria Bridgeman
Ms. Cheryl Coble
Ms. Debbie Coker
Ms. Carol Cook
Ms. Crystal Frazier
Ms. MariBeth Gagnon
Ms. Sharon Granade
Mr. Tom Hearn
Mr. Edwin Holmes
Dr. Dick Keenlyside
Dr. Katherine Kelley
Dr. John Krolak
Dr. John C. Ridderhof
Ms. Eunice Rosner
Dr. Shahram Shahangian
Ms. Elva Smith
Dr. Steve Steindel
Dr. Tina Stull
Ms. Julie Wasil
Ms. Rhonda Whalen
Mr. Mark White
Clinical Laboratory Improvement Advisory Committee
The Secretary of Health and Human Services is authorized under Section
353 of the Public Health Service Act, as amended, to establish standards to
assure consistent, accurate, and reliable test results by all clinical
laboratories in the United States. The Secretary is authorized under Section
222 to establish advisory committees.
The Clinical Laboratory Improvement Advisory Committee (CLIAC) was
chartered in February 1992 to provide scientific and technical advice and
guidance to the Secretary and the Assistant Secretary for Health regarding
the need for, and the nature of, revisions to the standards under which
clinical laboratories are regulated; the impact on medical and laboratory
practice of proposed revisions to the standards; and the modification of the
standards to accommodate technological advances.
The Committee consists of 20 members, including the Chair. Members are
selected by the Secretary from authorities knowledgeable in the fields of
microbiology, immunology, chemistry, hematology, pathology, and
representatives of medical technology, public health, clinical practice, and
consumers. In addition, CLIAC includes three ex officio members, or
designees: the Director, Centers for Disease Control and Prevention; the
Commissioner, Food and Drug Administration; and the Administrator,
Health Care Financing Administration; and such additional officers of the
U.S. Government that the Secretary deems are necessary for the Committee
to effectively carry out its functions. CLIAC will also include a non-voting
liaison representative who is a member of the Health Industry
Manufacturers Association and such other non-voting liaison representatives
that the Secretary deems are necessary for the Committee to effectively carry
out its functions.
Due to the diversity of its membership, CLIAC is at times divided in the
guidance and advice it offers to the Secretary. Even when all CLIAC
members agree on a specific recommendation, the Secretary may not follow
their advice due to other overriding concerns. Thus, while some of the
actions recommended by CLIAC may eventually result in changes to the law,
the reader should not infer that all of the advisory committee's
recommendations will be automatically accepted and acted upon by the
Secretary.
Welcome and Announcements
The meeting was called to order by Dr. Morton Schwartz, Clinical Laboratory Improvement
Advisory Committee (CLIAC) Chairman who welcomed the Committee members.
Self-introductions were made by all members of CLIAC, and Dr. Schwartz stated the role
and function of the CLIAC. Dr. Edward Baker, Director of the Public Health Practice Program Office,
Centers for Disease Control and Prevention (CDC), and Executive Secretary of CLIAC, made
additional welcoming remarks and announcements. Mr. Kevin Malone, Attorney Advisor, CDC,
reviewed the conflict of interest forms and format for disclosure of each committee member's
relevant financial interests as they relate to any topics discussed during the CLIAC meeting.
Presentations and Committee Discussion
CLIA UPDATE/CDC
CLIA Regulations/Archer Bill Addendum A
Dr. Carlyn Collins, Director of the Division of Laboratory Systems (DLS), presented a status
report on the Clinical Laboratory Improvement Amendments (CLIA) regulations. Dr. Collins
explained that revisions to the CLIA regulations pertaining to the moderate complexity
subcategory, physician-performed microscopy, and qualification requirements for general
supervisor and high complexity testing personnel were published as a final rule with comment in
the Federal Register on April 24, 1995. She further explained that the proposed rule to clarify the
waived criteria and the proposed rule to solicit comments on a new subcategory of moderate
complexity, accurate and precise technology (APT) tests, are currently under Department review.
Dr. Collins also briefly reviewed the Archer bill (H.R. 1386 and introduced in the Senate as
S.877). Dr. Collins explained that the Archer bill would amend the CLIA law to exempt physician
office laboratories (POLs) in which tests are performed as an adjunct to other services. POLs that
perform pap smears would not be exempt. In the bill, "physician" is defined by reference to the
Social Security Act which includes a Doctor of Medicine or Osteopathy, a Doctor of Dental
Surgery or of Dental Medicine, a Doctor of Podiatric Medicine, and a Doctor of Optometry.
Copies of the Archer bill were provided to the committee members.
Committee Discussion
Members questioned whether the revisions to the CLIA regulations published April 24, 1995
contained the same language as the CLIAC recommendations for changes to the CLIA
requirements. Explanation was given by CDC that the language in the April 24, 1995 regulation
was the same as the Committee recommendations with the addition of dates pertinent to the
personnel sections. A committee member asked if the CLIAC recommendations that were not
included in the CLIA rule published April 24, 1995 would be in the final regulation. Dr. Collins
responded that there were still a number of important areas to be addressed in the final, final
regulation. A committee member requested a list of the CLIAC recommendations and their
current status. This was provided to the committee members by CDC. (See Addendum B)
A committee member questioned why the Department would proceed to develop regulations for
APT categorization when the CLIAC had expressed reservations about the subcategory.
Dr. Collins explained that although there were previous discussions with the CLIAC on APT,
Committee input was not the only factor considered. Because of broad public interest, the APT
regulation has been developed as a proposed rule to allow increased opportunity for public input
and solicitation of comments. Dr. Baker pointed out that in previous discussions, CLIAC
indicated that the complexity model needed refinement. Publishing APT as a proposed rule would
allow the public to submit suggestions for revisions to the subcategory and revisions to improve
the complexity model. After the comment period, CDC could analyze the comments and at a
future CLIAC meeting, the comments could be summarized for Committee discussion.
Dr. Schwartz requested that a Committee meeting be scheduled to discuss the comments to the
APT proposed rule before publication of the final rule.
One committee member asked if the Archer bill would exempt POLs that accept referral
specimens. Dr. Collins explained that the bill could include POLs that accept referral specimens
and pathologists' laboratories, but that, at this time, it was unclear how the bill was to be
interpreted.
Review of the CDC Process for Waiving Tests Addendum C
A review and update of the process for evaluating applications for waiver was presented by
Ms. Rosemary Bakes-Martin, Laboratory Practice Standards Branch. Ms. Bakes-Martin
explained that the moratorium on adding tests to the waived category was lifted in December.
Letters were sent to manufacturers announcing the end of the moratorium and providing
guidelines for submitting requests for waiver. The guidelines followed the clarifications to the
waived criteria previously discussed and reviewed by CLIAC. In May, manufacturers were
notified that test systems cleared by the Food and Drug Administration (FDA) for home use
would be waived and were provided additional clarification in some areas of the guidelines.
Ms. Bakes-Martin described the process for reviewing waiver submissions, calling it a very
interactive process and stated that to date, ten manufacturers have requested waiver on 22
analyte/test system combinations. Two test systems have been granted waiver, Chemtrak
Accumeter (also known as Johnson and Johnson Advance Care) total cholesterol and HemoCue
glucose. The Chemtrak Accumeter test system was granted waiver based on FDA clearance for
home use. The HemoCue glucose test system met the CDC guidelines to the waived criteria and
was approved for waiver.
Committee Discussion
Committee members asked a few general questions about the waiver process, and Dr. Schwartz
explained that previously the Committee had been involved in reviewing CDC's clarifications to
the waiver criteria but had requested not to be routinely involved in reviewing individual test
systems that applied for waiver. Dr. Baker outlined the following as possible CLIAC roles: at
CDC's request, CLIAC may be asked to review clarifications to the statutory criteria for waiver;
provide oversight in the implementation process; and while CLIAC would not be performing
individual test system reviews, it may be asked to evaluate application of the guidelines to a
specific device.
Most of the members expressed some concern about the criteria used by the FDA for home use
clearance and the policy to waive under CLIA, tests cleared by the FDA for home use.
Committee members asked whether CDC conducts literature searches since some of the products
already cleared for home use have problems that have been well documented in various
publications. The Committee also asked whether this waiver review was considered an interim
process since the proposed rule to clarify the waived criteria had not been published. The
Committee then asked what would happen to those devices receiving waiver during this interim
period once the clarifications to the waived criteria are finalized. Ms. Bakes-Martin explained that
test systems approved for waiver would be published as a notice in the Federal Register with
opportunity for public comment and a test system's waiver status could change based on
comment analysis. It was also explained that there could be some revisions to the waiver process,
but any revisions to the process would probably not be major, and devices waived during this
interim period would probably meet the requirements in the final rule for waiver.
Ms. Bakes-Martin added that waiver is effective on the date of notification to the manufacturer.
A notification letter is also sent to the Health Care Financing Administration (HCFA). In
addition, the test categorization list, which is available on the Internet, is updated monthly.
Another member reiterated that when the Committee had previously discussed the three statutory
criteria for waiver, there had been agreement that the decision for waiver would be based on
performance and that some devices cleared for home use were poor performers and would not
meet the performance criteria. Dr. Collins responded that, in accordance with a CLIAC
recommendation, those devices previously waived would be subject to any new criteria when
finalized and this provision was included in the proposed rule for test waiver.
The Committee's manufacturer liaison asked if the waiver application was evaluated
simultaneously with the FDA review for new products and questioned the CDC's request for raw
data when the FDA only requires submission of raw data for clearance of Class III devices. CDC
stated that the two review processes are conducted simultaneously, and the manufacturer is
notified of the waiver decision once FDA clearance is obtained. CDC also explained that the
waiver review is focused on the performance studies and whether the specific waived criteria are
met. Many of the waiver requests have only included manufacturer's summary data which has not
been sufficient to evaluate the test system performance to verify that the waived criteria are met.
When the manufacturer provided raw data, the CDC was able to do the statistical analysis and
determine whether specific performance specifications were met. Ms. Bakes-Martin emphasized
that the CDC has been working closely with manufacturers and the waiver review process has
been going well. Again, the manufacturer liaison noted that the submission of summary versus
raw data was an issue and expressed concern about government resources being used to perform
statistical analysis when manufacturers should be doing their own statistical analysis. Another
committee member countered that requiring submission of raw data allows CDC to perform the
analysis needed and can be used to validate the manufacturers' summary data. One member
reiterated that, at a future meeting, it would be useful for the CDC to demonstrate how the
guidelines are applied to specific test systems.
Summary of Research Data and Analysis Activities Addendum D
Dr. Tom Hearn, Chief, Laboratory Practice Assessment Branch, presented some of the CDC
CLIA research and data analysis activities and noted that the data came from a variety of sources.
Demographic information was presented about the nation's clinical laboratories. Volume and
scope of testing information for specific laboratory types was also presented. Information on
laboratory practice including personnel data and quality assurance practices followed. The
presentation concluded with laboratory performance indicators based on proficiency testing (PT)
and inspection data, and a discussion of the numerous factors currently influenced by the practice
of clinical laboratory medicine.
It was announced that a clinical laboratory medicine research institute is planned for October of
this year entitled, "Frontiers in Laboratory Practice Research." The institute will provide a forum
for presentation of existing research and data in the clinical laboratory arena, and will include an
open discussion of the current research methods, agendas, and strategic planning for the future of
research in clinical laboratory medicine.
Committee Discussion
Several committee members requested clarification of the data pertaining to PT performance. The
CLIAC Chairman commented that the number of laboratory closures after the implementation of
the CLIA program was remarkably small in comparison to what has been reported by some
groups. CDC responded that this data and results of an Office of the Inspector General study did
not support the prediction that numerous laboratories would close due to CLIA. However, one
committee member noted that the data shows that 20% of POLs have reduced the amount of
testing performed. Dr. Hearn responded that a reduction in performance of a few microbiology
procedures probably accounted for the majority of the decreases in testing. Dr. Hearn added that
other factors may also have contributed to some reductions in testing services. For example,
shifts from solo to group practice, and shifts from fee-for-service practices to managed care,
practice guidelines, and financial considerations. Another committee member noted the
discrepancy between information included in the Regulatory Impact Analysis in the February 1992
CLIA regulations, which predicted that 15% of all laboratories would be waived, and current
HCFA certification data which shows 45% of all laboratories are waived. Ms. Judy Yost
reminded the Committee that the estimated number of waived laboratories predicted by the
Regulatory Impact Analysis was based on unverified assumptions as no data were available in
1992 to indicate either the number of laboratories in operation or the types of services performed
by these laboratories.
One committee member suggested that it would be valuable to have baseline data on the number
of unregulated laboratories versus regulated laboratories prior to CLIA implementation, and the
number of POLs compared to other laboratory types, to accurately ascertain if CLIA has affected
performance. Another committee member pointed out that although the data reflects that a
number of laboratories have PT failures, POLs have shown significant improvement in PT
performance over time, a finding which is consistent with the data recently published by the
College of American Pathologists (CAP). A committee member stated that the inspection data on
POLs presents a persuasive argument showing POLs can benefit from and should be subject to
regulation. One committee member noted that insurance companies are looking for an objective
standard such as laboratory accreditation or certification to serve as a quality indicator of
laboratory testing in POLs. Another committee member noted that exempting POLs from CLIA,
as proposed in the Archer bill, might not be necessary if the shift from solo or small group
practices to managed care continues.
One committee member noted that physicians do not initiate patient care and treatment based
solely on laboratory values; other factors such as the physical examination are also considered.
The Chairman suggested that when a patient is available for a direct evaluation by the physician,
an unexpected laboratory value should not be ignored. Instead, the laboratory value should alert
the doctor to reevaluate the patient's condition.
A committee member asked if any studies have been done to determine whether there were
improvements in patient outcomes as a result of quality improvements in laboratory testing.
Dr. Tina Stull, Laboratory Practice Assessment Branch, stated the CDC has a cooperative
agreement with the Ambulatory Sentinel Practice Network to determine the nature and frequency
of problems in the total testing process, and their impact on patient care. While the study is not
yet completed, early data indicates that most laboratory problems (60%) occur in the initial or
pre-analytic phase of testing, while 15% occur in the analytic phase of testing. Dr. Stull
commented that, in reality, the percentage of analytic phase problems may be higher but some
"analytic-type" errors may not be as apparent to the physician. In response, one committee
member commented that in a POL there should be less chance of errors occurring in the pre and
postanalytic phases of testing since specimens are generally collected and tested at the time of the
patient visit thereby eliminating errors during specimen transport and result reporting.
Overall, the Committee was impressed with the research studies and agreed this type of data is
critical for monitoring and evaluating laboratory practices, and the implementation of the CLIA
program.
CLIA UPDATE/HCFA
Summary of HCFA Survey Data Addendum E
Ms. Judith Yost, of the HCFA, presented an update on CLIA registration/certification, inspection
findings, and enforcement activities. At the completion of the first inspection cycle, 20% of the
laboratories had no deficiencies. The most common condition level deficiency was failure to
enroll in proficiency testing (59%) and the most common deficiency overall was failure to test two
levels of controls. Laboratory enforcement actions taken include the following: 8 certificate
limitations; 8 certificate suspensions; 11 certificate revocations; and 14 suspensions of Medicare
payments. HCFA continues to employ a consultative, educational approach to inspections to
assist laboratories in achieving compliance with the CLIA regulations.
Committee Discussion
A few committee members asked specific questions regarding the CLIA inspection data, such as,
were PT deficiencies due to failure to enroll in any PT program or failure to enroll in a PT
program for specific regulated analytes and which type of laboratories had no deficiencies?
Ms. Yost answered that condition deficiencies in PT included failure to enroll for any regulated
analyte for which the laboratory was performing patient testing or significant PT deficiencies; and
the breakdown of laboratories with no deficiencies were 17% POL, 25% independent, and 13%
hospital. It was noted that preliminary data on second cycle inspections indicate that laboratories
continue to show improvement in complying with the CLIA regulations, as evidenced by fewer
deficiency citations. Another member stated that the Commission on Office Laboratory
Accreditation (COLA) is developing a paper for publication that will discuss whether there is any
significant change in laboratory performance over time due to enrollment in an accreditation
program. The committee member added that, in general, COLA accredited laboratories inspected
during the second inspection cycle were showing significant improvement.
Proposal for a Performance-Based Survey Process Addendum F
Ms. Yost also presented some general information regarding a proposal to establish a
performance-based survey process which would be consistent with the Administration's
"reinventing government" efforts to ensure that regulations are appropriate and effective and
reduce the regulatory burden when possible. Additionally, it was noted that the total quality
management approach provides a rationale for improving the survey process and recognizing and
rewarding laboratories that have demonstrated compliance. Ms. Yost commented that this
process would be more outcome-oriented and more efficient since on-site surveys would be
focused on those laboratories with problems. Good laboratory performers would be recognized
and allowed to complete a questionnaire (off-site paper survey) in lieu of an on-site inspection.
She then asked the Committee for comments on this approach for improving the survey process.
Committee Discussion
A majority of the Committee expressed strong support and agreement with the suggested
approach to improve the existing CLIA inspection process. In particular, the Committee agreed
with focusing more on the outcome rather than process and rewarding laboratories that are good
performers. Members of the Committee asked the following specific questions: would the
laboratory be required to retain records for four years when the inspection cycle shifted from two
to four years and would the off-site paper survey be optional for the states or would all states
have to participate? Ms. Yost clarified that only one on-site inspection cycle would be skipped
and that no changes in the record retention requirements are anticipated; laboratories would
continue to maintain records for two years. In addition, any revisions to the process would have
to be consistent in all states and surveyors would need orientation training in the new process.
Committee members noted that the CAP and COLA accreditation programs have developed
questionnaires for off-site evaluation purposes. One member commented that this approach also
has been used for years in industry and worked well.
Other committee members suggested consideration be given to those laboratories having only
minor deficiencies rather than requiring laboratories to be 100% deficiency-free in order to be
eligible for the offsite survey. Also, one member cautioned against evaluating quality assurance
practices too rigorously until there are defined standards, while another member noted that care
must be taken to ensure that the off-site paper survey was not a "paper chase."
One committee member asked if financial incentives could be given as a reward for good
performance. Ms. Yost responded that currently there is a problem with the CLIA fees
supporting the program costs. The member then suggested redistribution of the financial burden
based on performance, which Ms. Yost agreed could be considered.
In general, the Committee agreed with the idea of providing laboratories incentive to reduce the
inspection burden but believed laboratories should continue to be monitored and an initial onsite
survey is needed as a baseline. Inspections should focus on outcomes and correlation with patient
results, and the requirements should be less burdensome but not at the expense of the patient.
REVIEW OF THE IMPLEMENTATION OF PERSONNEL STANDARDS Addendum G
A review of the history of the development of the personnel standards in the current regulatory
model was presented by Ms. Rhonda Whalen, Laboratory Practice Standards Branch. It was
emphasized that the requirements are interrelated and are balanced to establish minimum
standards of quality the laboratory must meet. The final regulations established personnel
standards appropriate to ensure that low volume, limited service laboratories would be able to
continue testing while large volume laboratories would be able to perform testing in a manner
suitable for that environment. However, the Department has received a wide range of comments
that the personnel regulations are either too stringent and need to be revised to, for example, only
require a qualified director who would be responsible for ensuring that laboratory personnel are
qualified; or too lenient with respect to general supervisor, and the requirements should be revised
to require, at a minimum, a bachelor's degree. At this point, the Committee was asked to
comment on whether the personnel requirements are, in general, appropriate.
Committee Discussion
One committee member asked how it was possible to establish the minimum standards required to
assure quality since there is limited data relating personnel qualifications to laboratory
performance. Ms. Whalen responded that in formulating the regulations, consideration was given
to public comments and input from consultants, but the variety of laboratories which would be
regulated under CLIA made it difficult to set minimum standards. Access to care had to be
considered and there was concern that if the qualification requirements were too high, they would
be impossible to meet and access to laboratory services would be limited.
One committee member questioned the rationale for the discussion since the Committee
previously had provided considerable input to the personnel qualifications and responsibilities.
Dr. Baker explained that it would be helpful for the Committee to provide some general comment
on the approach taken in establishing the personnel requirements, whether the approach was
appropriate and if laboratories are having difficulty meeting the personnel requirements. The
inspection data indicate that laboratories are in compliance so the assumption is that laboratories
do not find the requirements difficult to meet.
A committee member stated that the supply and demand issues which existed two years ago are
no longer applicable. In fact, the committee member noted that currently there is no shortage of
medical technologists. However, one committee member commented that in rural and urban
areas, public health laboratories have difficulty hiring qualified personnel to provide services to
patients who are uninsured and complying with the CLIA responsibility requirements creates a
hardship for directors of family planning clinics that have multiple laboratory sites. Another
member stated that in rural areas the costs associated with employing medical technologists was
an issue and that a person on-the-job trained was more desirable. A couple of committee
members stressed the negative impact the regulations have had on baccalaureate education and
training programs. Graduates of these programs can not find jobs, and schools have closed
because the regulations only require high school graduates or individuals with an associate degree
to perform testing. It was noted that lowering personnel standards results in people with
minimum education performing laboratory testing, jobs in laboratory services are no longer
attractive, and CLIA has played a role in this trend.
One committee member observed that competence is related to the environment and is situation-
dependent and noted that the current personnel standards are totally inadequate and do not even
provide a minimum standard for quality. She asserted that individuals who argue that standards
are unnecessary make unfounded assumptions, such as, the existence of a host of safe guards in
the system that negates errors in laboratory testing. She was puzzled why there continues to be
challenges to the need for standards in laboratory services when it is widely recognized that other
health professionals (i.e., registered nurses and physicians) have standards of competence
including education and certification requirements. The committee member noted that concerns
about personnel qualifications are financial and are not related to quality. Another member agreed
and stated that if financial resources were not involved, there would be no question about
compromising quality. One member stated that for performance of any laboratory test, personnel
need a basic knowledge of science and an understanding of what can go wrong during test
performance, and that the goal should be to improve test quality but this goal cannot be achieved
by continually lowering personnel standards to save money.
Another committee member stated that the focus needs to be on establishing the acceptable level
of performance, whether it is at a single level or multilevel, and determining what is necessary to
achieve that level of quality. He pointed out that there are laboratory activities that are simple,
push button, and require no judgment, but others require training and experience to make good
interpretations and decisions and these need to be identified. When testing requires higher levels
of knowledge, this type of testing is easily recognized, the problem is how one would assess the
level of knowledge required to operate an instrument. He also stated that if the personnel
requirements are going to be reconsidered, one must build a structure that defines the need,
specifies the demands or desired outcome, and defines the parameters or the bottom line that one
is willing to accept; then the educational infrastructure must be established to meet this need.
Several committee members commented that the "grandfather" provisions were appropriate but
noted that the general supervisor is the key to laboratory quality and the education requirement
should be set prospectively to require, at a minimum, a baccalaureate degree in a science. One
committee member noted that although the "grandfather" provisions have been added to the
regulations, the Department had not acted on the CLIAC recommendations to require general
supervisors in the future to have a baccalaureate degree; and in the committee member's opinion,
the current regulations allowing individuals with an associate degree plus two years training or
experience to serve as general supervisor are appalling.
Committee opinion varied on the minimum qualifications required for testing personnel. Two
committee members commented that the major problem is the law which requires site neutrality,
resulting in a "one size fits all" regulation. Many committee members felt that there are
problems with the complexity model and commented that both high and moderate complexity
have too large a range of tests and that along with test complexity, laboratory function needs to
be considered. Some committee members felt that the test complexity categories should be
multi-tiered.
Dr. Collins asked how the Committee felt about maintaining the general supervisor and director
requirements but allowing these individuals to choose the qualifications for testing personnel.
There was limited discussion about this proposal; however, committee members agreed that it
would be unwise to make any changes in the personnel requirements until longitudinal data on
laboratory personnel and performance are collected and analyzed.
CLIA REQUIREMENTS FOR QUALITY CONTROL Addendum H
Dr. Tom Hearn began with the provisions in the law related to quality control (QC) and then
presented background on how the QC requirements are balanced with other standards. The
phase-in of the QC requirements applicable to moderate complexity commercial test systems and
the minimum requirement of testing two levels of control every 24 hours were discussed. It was
noted that the guiding principles used to establish the CLIA requirements included: QC is the
process used to prevent and detect laboratory errors; the QC process must monitor environmental
conditions, specimen manipulation and testing performance; and QC materials (samples) should
meet the current National Committee for Clinical Laboratory Standards (NCCLS) definition. It
was also noted that most errors occur outside the testing (analytical) process. In inspected
laboratories, the most frequently cited deficiencies were related to the requirement to test two
levels of control and failure to follow manufacturers instructions.
Committee Discussion
Two committee members asked for the rationale for the quality control phase-in. Dr. Hearn
explained that the purpose of the phase-in was to allow previously unregulated laboratories time
to become familiar with the QC requirements and to allow FDA sufficient time to develop a
process to review manufacturers' QC instructions for compliance with CLIA QC requirements.
The member then asked how the FDA would establish this process since the FDA has no
resources for this purpose and questioned the reason for extending the phase-in when it appears
that the FDA will not be reviewing manufacturers' QC instructions for CLIA compliance.
Dr. Steve Gutman, of the FDA, responded that at this time the FDA has not established a review
process for clearing manufacturers' instructions for CLIA compliance. He stated that the FDA
continues to clear products under the 510(k) and premarket approval (PMA) processes.
Dr. Baker assured the Committee that the Department is aware of the problem and is looking at
other strategies including clarifying what types of QC laboratories need to perform and
recognizing the role of manufacturers in defining QC for devices.
The manufacturer liaison explained that FDA's 510(k) and PMA submission requirements
require manufacturers to provide information pertaining to the test system's labeling instructions
which include QC protocols. Prior to CLIA, the FDA's review of the test system's labeling
included any QC procedures that the manufacturer believed to be appropriate to monitor the test
system's performance. However, after CLIA implementation, the FDA has increased its scrutiny
of manufacturers' QC statements and is ensuring that the QC instructions are clear and specify
the portion of the test system monitored, and FDA does question QC instructions that are
inconsistent with the CLIA requirements. The manufacturer liaison concluded by stating that for
many products, the manufacturers' QC would be in conformance with the CLIA QC
requirements, for other products, particularly new technology, many manufacturers feel that the
CLIA QC requirements are excessive. In this committee member's opinion, the QC phase-in
extension was a disappointment to manufacturers because more flexibility needs to be added to
the CLIA QC requirements to accommodate new technologies since some of these devices do not
require six-month calibration or testing two controls each day of patient testing.
Dr. Gutman explained that under the Federal Food, Drug and Cosmetic Act, the FDA is
concerned with performance standards of products, but the emphasis in the 510(k) clearance
process is substantial equivalence to a predicate device (a product that has been cleared by the
FDA). The FDA has always evaluated a device's precision and bias in relation to a reference
method and in fact, the FDA may require that test system instructions include QC protocols that
are more stringent than CLIA if it is felt that more than two controls are needed to monitor test
system performance. While the FDA does not provide direction in terms of the frequency of
testing QC materials, it does determine whether the manufacturer states in its instructions the
portion of testing monitored by the control(s).
One committee member pointed out that the current QC standards were developed because
unregulated laboratories were opposed to the QC requirements and as a result, the QC standards
were "watered down" to accommodate site neutrality. The committee member noted that not all
test systems categorized as moderate complexity are kits and in fact, the moderate complexity
category includes many complex instruments such as CoulterT counters that require
sophisticated quality control and analysis. In this committee member's opinion, the list of
moderate complexity tests is too diverse for the minimum QC requirements that are in effect
during the phase-in, and the QC regulations are totally inadequate for the large sophisticated
instruments categorized as moderate complexity.
A committee member asked about the difference between moderate and high complexity testing
by suggesting that for moderate complexity tests, the burden is on the manufacturer to validate
the method as opposed to the requirement for the laboratory to verify or establish the
performance specifications for high complexity testing. CDC explained that during the phase-in,
laboratories performing testing using commercially available, unmodified moderate complexity
test systems are not required to perform a method validation. However, for all other testing, the
performance specifications must be established or verified by the laboratory before testing patient
specimens. Another committee member suggested that the CDC conduct a study that would
compare performance between laboratories that validate test systems and laboratories that follow
the minimum phase-in requirements and do not verify manufacturers' test system performance
specifications. The manufacturer liaison stated that allowing the laboratory to verify the
manufacturer's performance specifications in lieu of requiring laboratories to establish the test
performance specifications was working well and suggested that CDC initiate a retrospective
study to evaluate this process.
One committee member stated that the quality control requirements are minimal and with the
variety of test systems available, the laboratory has the responsibility for determining which test
methods are appropriate for the laboratory's test menu and, in addition, deciding on the best QC
practices to employ. The committee member suggested that instead of laboratories concentrating
QC efforts on individual test systems, kits or devices, the focus should be on the laboratory's
overall QC program.
CDC responded that when the QC requirements were developed, it was felt that QC should be
required for all testing and there should be no difference between the QC requirements for
moderate and high complexity. The QC phase-in was created to allow manufacturers time to
develop their QC protocols for FDA CLIA clearance but ultimately after the phase-in the QC
requirements would be the same for moderate and high complexity testing.
In reference to matrix problems, Dr. Schwartz asked if the QC materials were supposed to meet
NCCLS guidelines and if there had been an effort to encourage manufacturers to meet the
NCCLS standard. CDC responded that the intent was to have laboratories employ controls that
are as close as reasonably possible to the type of patient specimens tested. This elicited
discussion about the use of alternate control systems with some testing devices and new
technology. CDC asked for Committee opinion about the use of internal, procedural, and
electronic controls. Since many of these controls only monitor the analytical process, CDC asked
how the Committee felt the rest of the testing process should be monitored as required by the
regulations. While many committee members agreed that flexibility is needed for new
technology, not all "controls" are true monitors of the total testing process and these materials
can not substitute for quality control on some devices.
A committee member pointed out that the data presented earlier in the meeting indicated that the
majority of testing errors occurred in the pre and postanalytic phases of testing and consequently
to focus on the analytic phase is inappropriate and much too narrow. Monitoring the pre and
postanalytic phases have to be included in the laboratory's QC of the test system.
A committee member stated that the CLIA regulations include requirements for quality assurance
(QA) which address the pre and postanalytic phases of testing; however, he thought QA was not
included in the proposed APT subcategory. The committee member suggested using the term
total quality process instead of quality assurance. Another committee member agreed that quality
control on an APT instrument should be part of the laboratory's quality control monitoring
process. Dr. Collins responded by emphasizing that the proposed APT subcategory includes
requirements to monitor the total testing process i.e., QA, QC, and PT.
A few members stated that some previously unregulated laboratories did not know how to meet
the QA requirements which are too vague and allow too much flexibility leading to
misinterpretation, and these members indicated that more specific guidance is needed to assist
laboratories in meeting the QA requirements.
Returning to the QC discussion, Ms. Yost emphasized the dilemma for surveyors in evaluating
laboratories that use procedural controls to monitor testing. Based on the Committee discussion,
the total testing process needs to be monitored by testing QC. However, she pointed out that for
commercially available, unmodified moderate complexity testing, during the QC phase-in,
surveyors have been allowing laboratories to meet the QC requirements by performing whatever
QC the manufacturer recommends for the test system. These controls often only verify the
viability of reagents but do not monitor the total testing process. She asked the Committee for
input on whether this practice should continue or whether the entire testing process as defined by
NCCLS, should be monitored and should there be a difference between the QC requirements for
moderate and high complexity testing.
A committee member pointed out that the electronic checks for some devices monitor the process
of the instrument converting a sample signal to an electronic signal, however, this check does not
monitor the entire process. He added that the process of specimen extraction should be evaluated
but this varies by instrument or is system dependent. Ms. Yost agreed and pointed out that
testing could be affected by temperature, humidity, reagent expiration, etc., and if the control(s)
do not evaluate the whole testing process, there is no way of detecting problems with these
factors. These undetected problems could result in inaccurate test values regardless of the
electronic control checks.
One committee member suggested that the financial implications should be considered and cited
the example of single test unit strep tests in which testing a positive and negative control requires
separate test units. He also added that the detection of group A streptococcus is dependent on
proper specimen collection and questioned how one would monitor this testing phase.
Another committee member replied that quality control for specimen collection is included under
quality assurance which requires laboratories to have a mechanism to monitor the total process,
including the pre and postanalytic phases of testing. She added that surveyor training needs to
include how to evaluate each instrument and determine whether appropriate control mechanisms
are in place.
Another committee member stated that the focus should be on determining what part of the
testing process is not being monitored and the error detection rate of procedural or component
controls versus testing two levels of control that monitor all phases of testing. He noted that this
information should be provided before the Committee recommends changing any of the CLIA
QC requirements. This was seconded by the Committee chairman.
The Committee chairman stated that these quality control issues should be examined more
closely at another CLIAC meeting either by the full Committee or the QC subcommittee. He
suggested that QC requirements for new technology be a topic for a future meeting and requested
that scientific data pertaining to test system accuracy when testing is performed in accordance
with manufacturer's instructions be presented. Dr. Collins agreed that it was appropriate to get
advice from CLIAC on QC issues and again reminded everyone of the upcoming institute
"Frontiers in Laboratory Practice Research" in which these topics would be discussed.
COMMITTEE BUSINESS/PUBLIC COMMENTS
Presentation of Plaques for Outgoing CLIAC Members
Dr. Schwartz made the announcement that a few members of the CLIAC were ending their terms
of service and presented the following members with individual plaques commemorating their
service: Dr. Paul Bachner, Ms. Virginia Charles, Dr. Stanley Inhorn, and Dr. Kenneth Matthews.
Public Comments
Mr. Robert J. Slomoff, representing HemoCue, expressed concern regarding the policy on
granting automatic waived status to products cleared by the FDA for home use. Mr. Slomoff
noted that the focus of the review for waived categorization is quite different from the FDA
clearance process. FDA clearance for home-use is for home-use and these products are not
intended for use in a laboratory. He then stated that if FDA home use clearance means that a test
is waived under CLIA, then CDC and FDA should harmonize their requirements and have
similar standards.
Dr. Schwartz noted the Committee's concern that FDA clearance for home use results in
automatic waiver status. Dr. Gutman responded that the FDA is willing to consider changes to
the FDA review process. He agreed that the FDA process is more of a "truth in labeling" process
which is different from the CDC review process for waiver. FDA's focus has been to determine
whether performance of a device is equivalent to a device that has been cleared. The review
process for home use has been more contingent on the public health need and usefulness of the
device and not so much related to scientific justification. FDA does not look at random or
systemic error but whether total error matches intended use, whereas CDC looks more at the
statistics related to laboratory test performance. FDA would be willing to work with the CDC to
harmonize the processes but the outcome may be more strict or less strict and might not
necessarily satisfy the Committee.
Observing the incompatibility between the FDA home use clearance process and the waiver
review process, which includes the determination that there is an insignificant risk of erroneous
result for waived tests, a committee member expressed concern about the policy to waive test
systems cleared by the FDA for home use. Another committee member pointed out that in the
FDA review, public health is a concern for home use clearance, but if these home use test
systems are waived, it could be a public health crisis since these devices could be used in
intensive care units throughout the country, and waived tests are not subject to personnel or
quality control standards. She noted that the criteria for waiver must be as stringent as possible.
She also stated that those devices that are currently waived should be retrospectively evaluated.
Another committee member pointed out that although the FDA has very thorough labeling
requirements for home use devices and more latitude in the guidelines for clearance of these
devices, the guidelines used to clear devices for use in laboratories should be more stringent
because of potential errors and their impact.
Ms. Polly Cathcart, representing the American Society of Clinical Pathologists, expressed
general concern that the I-Stat device was under review for waiver and referenced specific areas
of concern using the CDC guidelines for test characteristics, storage, and accuracy.
One committee member stated that the examples of problems described in testing using the I-Stat
instrument are things a manufacturer's study may not show but could be seen in the laboratory.
Another member inquired how the public could find out which devices or test systems are under
review for waiver. Dr. Collins responded that the manufacturer may make the application public,
but the CDC kept the applications confidential. Dr. Schwartz stated that the CLIAC may need to
get involved in the review of multianalyte instruments. Dr. Collins also stated that any tests
granted waiver are published as a notice in the Federal Register which provides an opportunity
for public comment. A committee member pointed out that if the test was approved for waiver,
and then published in a Federal Register notice with a comment period, the device could be in
use for a long period of time before the comments are analyzed and any necessary changes are
made.
Toni Casey, a representative of I-Stat, responded that information on the precision and accuracy
of the I-Stat would be provided to the members.
One committee member stated that manufacturers were obligated to follow-up on complaints
about devices and if there are publications indicating inaccuracy, the manufacturer has to
investigate the claim and take action or notify the users of the device limitations. Another
member reiterated that the FDA and CDC review processes should be aligned and suggested that
one or both agencies report at the next meeting on how close the two processes are in reaching
alignment.
John Bruni, representing Biosite Diagnostics, commented that the waiver process should take
into consideration the intended use of the product and each individual product should be
evaluated according to intended use and not use guidelines that broadly apply to all devices.
David Phillips, representing Boehringer Mannheim Corporation, commented that some devices
are unitized, and that procedural and electronic controls should be acceptable QC for those test
systems. He noted that QC can be performed on a cartridge but the patient sample would be
tested with another cartridge. He gave the example of testing IV solutions for sterility by lot
number and stated the same principles could be applied to point-of-care instruments since the
manufacturer produces cartridges, strips, etc., in lot numbers. Dr. Schwartz asked what failure
rate is acceptable to the manufacturer. Mr. Phillips responded that there should be a 95%
confidence interval and recommended that laboratories perform a random check from each
shipment and lot number to verify acceptable limits.
One committee member agreed but questioned how this would be different from instruments
used in non-point-of-care sites. Another committee member stated that there was no difference
and that the issue was not about validating individual cartridges but looking at the entire testing
process. If each step in the testing process is identified, then you can evaluate which parts of the
test system need to be monitored based on the conditions that affect testing.
A committee member stated that in order to determine failure rate, he thought failure would need
to be defined. There is a cost issue when dealing with unitized devices and suggested that when
a shipment of reagents arrives in the laboratory, a full scale QC should be performed. After this
initial check, he thought that the shipment of reagents should be reliable and further QC of the
test system should not be necessary. However, he noted under CLIA, QC is required each day
patients are tested.
Conflict of Interest
Each committee member disclosed his/her financial interest(s) relevant to the topics discussed
during the meeting.
Summary
In closing, Dr. Baker summarized the Committee's suggestions and briefly outlined possible
topics for future CLIAC meetings:
The Committee should be provided an opportunity to discuss the comments received to
the APT proposed rule before publication of the final rule.
CDC should ensure that reviews of waiver requests are consistent and accurate and at
CDC's request, CLIAC would review any proposed clarifications to the statutory criteria
for waiver, provide oversight in the implementation process and evaluate application of
the waiver guidelines to a specific device;
The CDC and FDA should work towards achieving greater congruence between CDC's
review process for waiver and FDA's home use clearance process;
CLIAC should play an advisory role in reviewing the QC regulations and any revisions to
the CLIA QC requirements as they apply to new technology and could consider the issue
of test specific QC requirements versus requirements for an overall laboratory QC
program;
CLIAC may examine and discuss the effects, if any, of managed care on laboratory
services;
CLIAC supports deferring any recommendations for revisions to the personnel
qualification requirements until data on laboratory personnel and performance are
collected and analyzed; and
Information pertaining to current genetics testing practices should be presented to the
CLIAC to determine what recommendations, if any, the Committee should make in this
area of laboratory testing.
Future Meetings
Dr. Schwartz asked that the committee members consider dates for the next two CLIAC
meetings. The members agreed to schedule full committee meetings on August 30-31, 1995 and
January 24-25, 1996.
Send questions and comments to: Webmaster@DLS
|
