Table of Contents
I. Record of Attendance
II. Welcome and Announcements
III. Topics
CLIA Update
Centers for Disease Control and Prevention
Health Care Financing Administration
Overview of Comments to Cytology Proposed Rule
Presentations on Automated Instruments for Pap Smears
AutoPap
Papnet
CDC Cooperative Agreements on Computer-Based Cytology Proficiency Testing
American Society of Clinical Pathologists
Thomas Jefferson University
New England Medical Center
Solicited Public Comments
IV. Public Comments
V. Concluding Remarks
VI. The Addenda
Record of Attendance
The Clinical Laboratory Improvement Advisory Committee (CLIAC) met at the
Centers for Disease Control and Prevention (CDC), Auditorium B, in Atlanta,
Georgia, on March 6, 1996. Those in attendance are listed below:
Committee Members Ex Officio Members
Dr. J. Scott Abercrombie Dr. Carlyn Collins, CDC
Dr. Thomas Bonfiglio Dr. Steve Gutman, FDA
Ms. Michele Best Ms. Judith Yost, HCFA
Dr. Ronald Cada
Dr. Susanne Gollin
Dr. Verlin Janzen Executive Secretary
Ms. Sandra Johnson Dr. Edward Baker
Dr. J. Stephen Kroger
Dr. Bereneice Madison
Dr. Deborah McHugh Liaison Representatives
Dr. Wendell O'Neal Dr. Fred Lasky (HIMA)
Dr. Glenda Price
Dr. Sharon Radford
Dr. Patricia Saigo
Dr. Morton Schwartz
Mr. Elliott Segal
Centers for Disease Control and Prevention
Ms. Diane Bosse
Ms. Cheryl Coble
Ms. Debbie Coker
Ms. Carol Cook
Ms. MariBeth Gagnon
Dr. Richard Keenlyside
Mr. Tommy Lee
Dr. John C. Ridderhof
Mr. Gregory Smothers
Dr. Tina Stull
Ms. Julie Wasil
Ms. Rhonda Whalen
Clinical Laboratory Improvement Advisory Committee
The Secretary of Health and Human Services is authorized under Section
353 of the Public Health Service Act, as amended, to establish standards to
assure consistent, accurate, and reliable test results by all clinical
laboratories in the United States. The Secretary is authorized under Section
222 to establish advisory committees.
The Clinical Laboratory Improvement Advisory Committee (CLIAC) was
chartered in February 1992 to provide scientific and technical advice and
guidance to the Secretary and the Assistant Secretary for Health regarding
the need for, and the nature of, revisions to the standards under which
clinical laboratories are regulated; the impact on medical and laboratory
practice of proposed revisions to the standards; and the modification of the
standards to accommodate technological advances.
The Committee consists of 20 members, including the Chair. Members are
selected by the Secretary from authorities knowledgeable in the fields of
microbiology, immunology, chemistry, hematology, pathology, and
representatives of medical technology, public health, clinical practice, and
consumers. In addition, CLIAC includes three ex officio members, or
designees: the Director, Centers for Disease Control and Prevention; the
Commissioner, Food and Drug Administration; the Administrator, Health
Care Financing Administration; and such additional officers of the U.S.
Government that the Secretary deems are necessary for the Committee to
effectively carry out its functions. CLIAC will also include a non-voting
liaison representative who is a member of the Health Industry
Manufacturers Association and such other non-voting liaison representatives
that the Secretary deems are necessary for the Committee to effectively carry
out its functions.
Due to the diversity of its membership, CLIAC is at times divided in the
guidance and advice it offers to the Secretary. Even when all CLIAC
members agree on a specific recommendation, the Secretary may not follow
their advice due to other overriding concerns. Thus, while some of the
actions recommended by CLIAC may eventually result in changes to the law,
the reader should not infer that all of the advisory committee's
recommendations will be automatically accepted and acted upon by the
Secretary.
Welcome and Announcements
The meeting was called to order by CLIAC Chairman Dr. Morton Schwartz. The
committee members made self-introductions and disclosure statements of their
relevant financial interests as they relate to any topics to be discussed during the
CLIAC meeting. Dr. Schwartz then stated the role and function of CLIAC.
Dr. Edward Baker, Director of the Public Health Practice Program Office, CDC, and
Executive Secretary of the CLIAC welcomed the committee members. He noted the
celebration of CDC's fiftieth anniversary in 1996, preparation for the Olympics to be
held in Atlanta, and the upcoming opening of the CDC World Learning Center.
CLIA UPDATE/CDC Addenda A-E
Dr. Carlyn Collins, Director of the Division of Laboratory Systems, CDC, referred to
instructions distributed to Committee members for Internet access to Federal
Register publications, an updated chronology of CLIA-related Federal Register
publications, and the status of CLIAC recommendations as of March 1, 1996 (see
Addendum A-C). She noted that action on certain CLIAC recommendations is
pending the development of a final, final regulation, which may be published by the
end of the year.
Dr. Collins reviewed the comments and concerns of 43 letters (320 comments)
received in response to the publication of the waived proposed rule (see Addendum
D). She stated that, for the past year, CDC has been accepting requests for waiver
based on guidelines in the proposed rule and listed the test systems waived under
these guidelines. Then Dr. Collins reviewed the content of 51 comment letters (304
comments) received in response to the publication of the accurate and precise
technology (APT) proposed rule (see Addendum E).
Mr. Kevin Malone, Attorney-Advisor, CDC, presented the background for an appeal
by the Department of Health and Human Services (DHHS) to have a court ruling
by U.S. District Judge Gladys Kessler reversed. On two of four points, the judge
ruled in favor of the plaintiffs, Public Citizen and Consumer Federation of America,
stating that (1) the CLIA criteria for test categorization failed to consider the risks
and consequences of erroneous results to patients, and (2) proficiency testing (PT)
for cytologists does not reflect "normal working conditions". DHHS feels that the
current regulation takes risks into account and that test categorization based on the
potential consequences of erroneous results would be too subjective and
unworkable. In addition the Department believes that, to the extent practicable,
"normal working conditions" are incorporated in the cytology PT requirements in
the current regulation. DHHS appealed the court ruling and hopes to have a
decision by the end of the year. As required by the court ruling, DHHS published a
cytology proposed rule in November 1995.
Committee Discussion
One CLIAC member asked for clarification on the effective date of waiver for a test
system. Dr. Collins responded that waived categorization is effective when the
manufacturer is notified, which precedes the publication of a Federal Register
notice announcing waiver approval. Another committee member asked if any
requests for waiver have been denied. Dr. Collins responded that 44 requests have
been received and none have been denied to date, but not all waiver submissions
have been approved. She noted, however, CDC, in evaluating waiver requests,
works closely with the manufacturers. In addition, Dr. Collins commented that the
criteria for waiver approval by CDC are similar, but not identical, to the criteria for
home use approval by the Food and Drug Administration (FDA). Dr. Schwartz
stated that the problem is that, if the manufacturer of a device approved for home
use applies for waiver, CDC must approve the request for waiver. He then said
that he hoped that CDC and the Food and Drug Administration (FDA) were
working together to streamline the waiver approval process. Then Dr. Schwartz
asked Dr. Collins if CLIAC should revisit the guidelines for waiver. Dr. Collins
responded that the proposed clarifications to the criteria for waiver (as included in
the CDC guidelines and published in the waived proposed rule) appear to work
well, noting that the guidelines are objective and easy to apply.
With respect to APT testing, Dr. Schwartz remarked that CLIAC did not
unanimously support the proposed APT subcategory. Several other committee
members noted the previous CLIAC discussions that the establishment of the
proposed APT subcategory would not provide sufficient regulatory relief to
laboratories and should be reconsidered. Another member asked if the
subcommittee on test categorization would have further opportunity to discuss APT.
Dr. Collins replied that we anticipate additional discussions with the Health Care
Financing Administration (HCFA) and within the Department.
One committee member asked how the result of the appeal process would affect the
content of the cytology final rule. If the ruling of the lower court is overturned,
Mr. Malone suggested that the current regulation would remain in place. However,
if the decision of the lower court is upheld, DHHS will probably reevaluate options
for cytology PT. Mr. Malone noted the inability of the Department to implement
glass slide PT on a national scale, and said that responses to the cytology proposed
rule indicated that computer-based PT (CBPT) might be an alternative testing
media.
CLIA UPDATE/HCFA Addenda F-G
Ms. Judith Yost of the HCFA presented a status report on CLIA implementation
and a summary of the types of deficiencies sited during surveys(see Addendum F).
She noted consistency between the types of deficiencies cited in first and second
cycle surveys, i.e. failure to enroll in PT continues to be the most frequently cited
deficiency, followed by failure to perform or document quality control (QC)
activities, and absence of a quality assurance (QA) plan. Approximately 25-30%
fewer deficiencies in quality-related areas were cited in second cycle surveys. Ms.
Yost attributed the decrease in deficiencies to HCFA's educational approach in
conducting inspections. Physicians' office laboratories continue to have high
numbers of deficiencies, especially failure to perform and document two levels of
QC testing each day of use and failure to follow the manufacturer's test system
instructions.
As a part of Vice-president Gore's program to reinvent government, HCFA recently
began a performance-based survey process, in lieu of on-site inspection.
Laboratories with good performance on their first cycle survey (approximately 10-15%
excluding cytology laboratories) and satisfactory PT results are eligible for
evaluation by a self-assessment questionnaire (Alternative Quality Assessment
Survey) for one two-year cycle. Random on-site surveys will be conducted of the
laboratories receiving a performance-based survey. All laboratories will be
surveyed on-site at least every four years.
A revised on-site survey process, which focuses on quality outcomes, is also being
initiated. A surveyor will begin the inspection by reviewing the laboratory's QA
program. If the observed outcomes are good, the laboratories will be in compliance
with CLIA regulations. If problems in QA are detected, the surveyor will more
extensively evaluate the laboratory operation to determine areas of non-compliance.
Survey instructions for the process are currently being developed and training of
surveyors will begin in May. The new process is expected to be more efficient and
effective.
Nineteen proficiency testing programs have been approved for 1996. Oregon has
been approved as a State exempt program. Interest in State exemption has been
expressed by California and Georgia.
Ms. Yost commented that HCFA has received a number of questions concerning the
applicability of the CLIA regulations to the AutoPap 300 and the Papnet,
automated cytology devices recently approved by the FDA. She reviewed the
intended use of each device and stated that laboratories using these devices will be
in compliance with CLIA regulations if they follow the manufacturers' instructions
(see Addendum G).
Committee Discussion
Several committee members voiced concerns about HCFA's statistical data. The
following potential problem areas were noted:
1. For a given laboratory, the laboratory's type is self-reported and may be
incorrect.
2. Inspection data on accredited laboratories (61% of the nation's laboratories)
are not included in the laboratory statistical reports developed by HCFA.
Most hospital and independent laboratories have an accreditation certificate
and are not included in the HCFA database. Accrediting organizations such
as the College of American Pathologists (CAP), the Joint Commission on the
Accreditation of Health Organizations, the Commission on Office Laboratory
Accreditation (COLA), and exempt states (New York, Washington, and
Oregon) maintain their own data and information about laboratories that
participate in their programs.
3. Data is not up-to-date.
4. Data included in the table of the top 20 deficiencies is cumulative. If a
laboratory was cited for the same deficiency in its first and second cycle
surveys, the deficiency and the laboratory were counted twice.
5. No deficiency data is available on laboratories that have a certificate of
waiver.
6. Data on enforcement actions for 1995 is not yet available.
Ms. Yost agreed that there are a number of problems with the data, but said that
she anticipates having better data by the next CLIAC meeting, as well as a list of
laboratories cited for immediate jeopardy. She stated that accreditation
organizations are required to report enforcement actions to HCFA and that each
year HCFA performs validation survey of 3-5% of accredited laboratories; she
noted, however, that the requirements of accreditation organizations must be
equivalent but not identical to HCFA requirements. Dr. Kroger reported that
COLA's data also showed a decrease in the number of deficiencies cited during the
second cycle inspections, and the deficiencies cited are similar to those cited by
HCFA.
In response to concerns expressed earlier by several committee members,
Dr. Rabinowitz confirmed that the waiver and home use approval processes do not
mesh exactly. Several committee members commented that the intended use of a
device waived under CLIA is often quite different from a device approved for home
use and were concerned about the safety and effectiveness of a home use device
(such as a glucose monitor) being used in a non-home setting (such as a nursing
home). Dr. Rabinowitz explained that the FDA's responsibility is to review
submissions from the manufacturers based on the manufacturer's claim, i.e. they
review data on the intended use of a device to show its safety and effectiveness.
Dr. Schwartz again commented that the law suggests that the CDC must waive a
home use product if the manufacturer applies for waiver status. Another committee
member suggested that the CDC should require data to substantiate the safety of a
home use device when used in a non-home setting.
OVERVIEW OF COMMENTS TO CYTOLOGY PROPOSED RULE Addendum H
Ms. Rhonda Whalen summarized the comments received to the proposed rule to
revise the time frame for conducting cytology PT. Of the 757 comment letters
received, almost all were opposed to the proposed rate change. Almost 50% of the
letters provided comments on computer-based proficiency testing (CBPT). It was
noted that the numbers may change since CDC continues to evaluate comments.
Ms. Whalen stated the most common reasons given for opposition to the rate change
and listed suggested alternatives to the proposed PT time frame. With respect to
PT simulating workplace testing, commenters reiterated the differences mentioned
in the preamble to the proposed rule, and noted that three states require a lower
maximum daily workload than the maximum workload rate in the CLIA regulation.
Commenters also expressed concerns about the feasibility and validity of a national
cytology glass slide PT program, and suggested that the regulations be revised to
approve the glass slide programs that are currently available. Approximately 25%
of those who commented on CBPT were opposed, while the remainder were
supportive or apprehensive about such a program. Ms. Whalen discussed the
commenters' concerns and suggestions for implementation of a CBPT in cytology.
Committee Discussion
One committee member asked if cytotechnologists or pathologists supported CBPT
as an alternative to glass slide PT. Ms. Whalen replied that the data were not
analyzed in that manner, but that the data analysis is not yet complete. Another
member asked about the number of comments from consumers and Ms. Whalen
stated that no commenters identified themselves as consumers. Still another
member asked if the publication of a final rule will be delayed until the outcome of
the appeal process is known. Ms. Whalen noted that some of the comments to the
proposed rule have helped in preparation of a response to the court order.
Dr. Baker responded that the development and publication of a final rule takes
several months, so publication is unlikely to occur prior to a decision in the appeal
process.
PRESENTATIONS ON AUTOMATED INSTRUMENTS FOR PAP SMEARS
AutoPap Addendum I
Dr. Tom Anderson first explained the theory and operation of the AutoPap 300
approved by the FDA to assist in laboratory QC in cytology. He then described
components of the AutoPap system and the use of the instrument, noting that the
AutoPap contains a comprehensive quality assurance system integrity device to
assure that the instrument is constantly in calibration. Dr. Anderson reported
that, in the clinical evaluation of 12,000 slides, the instrument detected 34% of the
false negatives in all diagnostic categories and more than 50% of the false negatives
in the categories of low and high grade squamous intraepithelial lesions and cancer,
when compared to manual rescreens of all negative slides. Eventually, the
company will request FDA approval of the instrument for cytology PT and for
primary screening of Pap smears. Dr. Anderson asked the CLIAC to consider
revisions to the CLIA regulations appropriate for this type of technology and
proposed specific language to revise the CLIA regulations at 493.1257(d)(1)(I) as
follows:
The review must include negative cases selected by an automated
cytology device approved by FDA for use in quality control or at
random from the total caseload and from patients or groups of patients that
are identified as having a high probability of developing cervical cancer,
based on available patient information.
Papnet Addendum J
Dr. Laurie Mango described the intended use of the Papnet in the clinical setting.
All Pap smears categorized as negative by manual screening would be referred to a
Neuromedical Systems, Inc. regional scanning center. Smears would be rescreened
by Papnet and the most suspicious 128 images from each smear would be digitized.
The digital tape and the original slides would be returned to the laboratory, where
the digital images would be examined at a Papnet review station by
cytotechnologists. The original smears would then be evaluated microscopically as
necessary. Papnet thus assists in human decision making, i.e., it locates
potentially abnormal cells, but requires that the cytotechnologist or pathologist
make the final interpretation.
When the combination of manual screening with Papnet review of slides was
compared to manual screening alone in clinical trials, abnormality was found in
over 4.8% of smears routinely identified as normal at these clinical sites, i.e., a 30%
increase in relative sensitivity. The instrument is highly effective for the detection
of low numbers of abnormal cells.
Committee Discussion
Committee members asked the manufacturers for clarification of the criteria for
false negatives in the clinical trials of both instruments. In some instances in both
studies, the criteria for false negative Pap smears was based on clinical outcomes,
i.e. biopsy confirmation of disease. More often, however, that data was not
available. A few committee members were very concerned about the high false
negative rates reported in the studies (20-25%), and agreed that this statistic
should be related to patient outcomes, not just to laboratory results. The CLIAC
chairman noted that "worst case" published data indicated a false negative rate of
about 11%. Another committee member commented that ASCUS (atypical
squamous cells of undetermined significance) is responsible for a large component
of the false negatives.
Several committee members wanted information on the impact of the instruments
on turn-around-times and workflow in the laboratory. Since the AutoPap is located
in the laboratory and QC slides can be loaded on the instrument as soon as manual
screening is completed, there should be little effect on turn-around-times.
Laboratories using the Papnet must ship the slides to a regional center which scans
the slides, makes digital tapes, and ships the slides and tapes back to the
laboratory. The turn-around-time for this is 3-5 working days. In the laboratory,
cytologists then examine the digital images at a review station and select the slides
requiring manual review.
Committee members asked specific questions about the operation of Papnet.
Dr. Mango briefly described the "training" of the computer with normal and
abnormal images, scoring the cells, the reproducibility of the cell selection process,
and noted that high risk false negatives should be screened manually. Several
committee members asked about the cytotechnologists' review of the digitized
images. Dr. Mango responded that a cytologist should not be biased by the order of
the images presented (they are displayed according to geographic location, not
according to hierarchy of abnormality). Although the same number of images is
presented from every slide, the cytotechnologists in the trial studies selected about
25% of the slides in the control cohort for human review. Therefore, if a laboratory
rescreened 100% of its negative slides with Papnet, this would represent
approximately a 15% increase in workload above the 10% required for random
rescreen. Another committee member asked if the manufacturer claimed that
Papnet is more accurate than a 100% manual rescreen. Dr. Mango clarified that,
based on the data from a routine 10% CLIA mandated rescreen, the manufacturer
predicted that Papnet would detect more abnormal smears. The same committee
member felt that the manufacturer should not make this claim unless Papnet
results were compared to the results of a 100% manual rescreen.
The Health Industry Manufacturers Association (HIMA) liaison noted that there
had been a
5-month delay in HCFA's review of the AutoPap for CLIA compliance. He
commented that problems of this nature can delay laboratory use of the instrument
and are very expensive for the manufacturer. He was in support of the proposal by
Dr. Anderson to change the CLIA regulations to increase flexibility and allow FDA
and HCFA to work together more efficiently. Ms. Yost acknowledged that the
furlough of federal workers contributed to the delay, but stated that in September
HCFA and CDC requested additional information from the manufacturer which
was not received until December.
There was a discussion about the perception of the use of computer-based images in
cytology. One committee member said that the public's major concern about
cytology testing is the false-negative rate. He felt that, although the public might
be skeptical initially about computer-based screening of patient slides, this
perception could be improved through education that the false negative rate could
be lowered by using computer-based screening. Another committee member stated
that some of the public already believe that instruments make fewer errors than
humans. Several committee members were concerned about the negative reactions
of cytology personnel to CBPT. One member felt that the apprehension was
understandable, when individuals' jobs are at risk. Another member noted that
people lose their apprehension over time, while another CLIAC member felt that
recently trained personnel would be more receptive to CBPT, since they have been
exposed to computer-based certification examinations.
CDC COOPERATIVE AGREEMENTS ON CBPT IN CYTOLOGY
Dr. Collins briefly reviewed the requirements for cytology PT. She noted the
Department's inability thus far to implement a nationwide glass slide proficiency
testing program and the pending court case. As a result of discussions with CLIAC
in December 1993 concerning alternatives to glass slides (see CLIAC
recommendations in Tab E), CDC has pursued CBPT as an option to glass slide
testing.. Computer-based testing would provide uniform testing, require fewer
glass slides, have options for test administration, promote equitable testing, and be
less expensive. She noted that the major difficulties encountered are the inability
to adequately test locator skills, requirements for large amounts of computer
storage space, inability to test under normal working conditions, and uncertain
acceptance by the professional community. Dr. Collins stated that CDC has
completed three cooperative agreement studies which have explored the feasibility
of CBPT. Ms MariBeth Gagnon of the CDC explained the requirements of the
cooperative agreement and introduced the presenters from the American Society of
Clinical Pathologists, Thomas Jefferson University, and New England Medical
Center who presented results of the pilot studies.
American Society of Clinical Pathologists (ASCP) Addendum K
Ms.Theresa M. Somrak, JD, CT(ASCP), referred to the regulatory requirements for
cytology PT and commented that, because of the nature of testing, normal cytology
working conditions cannot be duplicated. However, she emphasized that cytology
workplace skills would be duplicated under testing conditions. Ms. Somrak
described the establishment of a bank of digitized challenges, the development of
software to administer the test, field testing of the images, and pilot testing
(including an opinion survey of the participants) at two scientific meetings. Then
Mary E. Lunz, Ph.D., discussed reasons for using CBPT, including validity and
reliability issues. She expressed concerns about the reliability of a test that
includes only ten challenges (as required for glass slide testing) and suggested that
increasing the number of challenges in CBPT would decrease the error of
measurement and therefore increase the precision and reliability of the test. She
then presented the results of the ASCP/CDC pilot study. She said that there was a
higher success rate on the still images than on the scan images of both
cytotechnologists and pathologists; she added that the reliability was low for both
the still and scan images, which she attributed to the small number of challenges.
When individuals' performance on CBPT was compared with their performance on
glass slide PT, 73% either passed or failed both tests, while 27% had discrepant
pass/fail results. Dr. Lunz stressed increasing the reliability of the test and felt
that too much emphasis was being placed on the mode of administration. She then
recommended that computer-based images be used in cytology PT. She said that
CBPT could provide a broader range of challenges than glass slide PT, could
adequately test individual competency, should occur in a controlled environment,
and concluded that testing under these conditions would approximate normal
working skills/responsibilities.
Thomas Jefferson University (TJU) Addendum L
Ms. Shirley Greening described the design and results of the TJU/CDC cooperative
agreement study. In the pilot study, each individual took a different test. Attempts
were made to simulate normal working conditions, i.e., the ability to examine each
field at low and high powers and to return to previous challenges. Although
participants were allowed 2 hours, most completed the test in 25-30 minutes.
Ms. Greening expressed disappointment in the results of the study. Only 54% of
the individuals who participated in the CBPT passed the test, compared to 80% of
those who participated in the glass slide PT. Many participants commented on the
poor quality of the images (vendor supplied an unsuitable monitor). Some
participants who failed had difficulty reading the text on the screen, said that the
selection of cells was inadequate to answer questions, and felt that the test was not
a reflection of their interpretive skills. Most individuals suggested that the ability
to focus up and down on still images would be helpful. Ms. Greening concluded
that the TJU system could currently be used for educational purposes and is a
viable alternative to glass slide PT. She noted that further development of certain
aspects of the TJU system is needed, and that comparison studies would be
necessary to determine valid and reliable performance benchmarks.
New England Medical Center (NEMC) Addendum M
Ms. Gagnon apologized to NEMC that CDC was unable to provide compatible
computer equipment for a demonstration. Dr. Martha Hutchinson described the
design of the NEMC system, which includes focus fields and movie loops. Dr. David
Zahniser presented the results of pilot studies performed at the ASCT and ASCP
meetings and later at local laboratories in the Boston area. Using ten CBPT
challenges, 17% of the participants failed the test. For 37 participants in both the
CBPT and the glass slide PT, the pass/fail results were in agreement, while 11
individuals had discrepant results. In general, the participants liked CBPT as a
testing method, were impressed with the quality of the computer images, and liked
the ability to focus. Participants suggested that the clinical history should remain
on the screen, and that improvements should include more and larger images, more
high power fields, strict Bethesda report groupings, and a light setting option.
Dr. Zahniser noted that the failure rates of NEMC and ASCP were similar, and
suggested that a computer-based test should contain 20 slides. He pointed out that
the NEMC system monitored the number and location of clicks made by the
participants. Individuals with less than five years of experience clicked more often
on a challenge and required longer examination time than individuals with more
than five years of experience. With further development of this feature,
Dr. Zahniser felt that the NEMC system could evaluate locator as well as
interpretive skills.
Committee Discussion
The CLIAC chairman stated that the issues for discussion were (1) the automated
instruments that have been approved by the FDA for cytology QC and (2) the use of
computer-based images in cytology PT. He noted the potential use of the automated
instruments in PT and commented that changing the language of the regulations,
as suggested by Dr. Anderson, would be a long, laborious process .
A committee member asked about the future direction of CDC's efforts in the area
of cytology PT. Dr. Collins said that the direction is currently undecided. However,
she noted that rule making is in process to revise regulations pertaining to PT
workload rate and stated that CDC had previously considered revising the CLIA
regulations to allow approval of a variety of PT methods. The same committee
member thought that results of the cooperative agreement studies were valuable,
and suggested that CDC enhance "the best of all three" with additional technology,
e.g., the use of CD ROMs for storage. In addition, she suggested that there should
be 20-30 challenges, instead of 10. Another committee member agreed, adding that
the number of challenges should be increased for any PT methodology.
Yet another committee member commented that she felt evaluating only the locator
skills of cytotechnologists is adequate, since the abnormal cells would then be
referred to a pathologist for interpretation. She asked for and received clarification
of the differences in grading the performance of cytotechnologists and pathologists.
Dr. Collins commented that poor locator performance may reflect a lack of attention
instead of a lack of skill. The same committee member asked about possible bias in
the selection of participants in the pilot studies. The presenters commented that
anyone who volunteered at the professional meetings was tested, but acknowledged
that the results could be biased toward individuals who attend regional meetings.
Another committee member asked about the possibility of taking a CBPT on-site by
simply logging into a central computer if the laboratory has the necessary computer
equipment. One of the presenters responded that, while it may be possible, the
logistics of administering a computerized test are quite complex. She proposed as
an alternative, using existing testing centers across the country, where testing
could be monitored, or suggested simply mailing and returning a disk if one is not
concerned about monitoring. A committee member, who had participated in the
ASCP pilot study, commented that she adapted easily to the testing method, and
felt that the perception of computerized testing would not be an issue. She asked
about the possibility of using a minimum number of slides to detect competent
individuals, and an increasing number of slides (with a maximum limit) to detect
incompetent individuals, as is done with the ASCP Board of Registry. The ASCP
presenter felt that this would be an option with CBPT, as would increasing the
standard number of challenges. She said that either option would increase the
reliability of the test. Another committee member, who agreed that adaptation to
computerized testing is not an issue, felt that CBPT is currently the only viable
option to glass slide PT, and that it could definitely improve the reliability of
testing.
Solicited Public Comments Addenda N-O
1. Dr. Mary Nielson read a statement (see Addendum N) reflecting CAP's views
on computer-based programs for cytology PT and recommendations for
changes in the current CLIA regulations, and described the CAP PAP PT
program. The committee chairman asked Dr. Nielson if any of the CLIAC
presentations would have affected the CAP position statement. Dr. Nielson
responded that the CAP's position, that glass slides are the best approach for
PT, would not have changed.
2. Kathy Grant, Ph.D., representing ASCP, described her research in computer-
based PT in cytology (see Addendum O). Based on the results of her
independent study, she recommended constructing a field-tested bank of
challenges using existing computer technology and increasing the number of
challenges to 25-50 to improve the reliability of the testing method. She
noted that her conclusions were similar, but not identical, to those of the
ASCP.
Committee Discussion
One CLIAC member was concerned that since the three pilot studies are completed,
CDC's involvement in CBPT would end. He recommended that CDC continue to
pursue the development of CBPT and another committee member agreed.
Dr. Collins clarified that although the pilot studies are completed, CDC has a 2-year
contract to study the actual work performance of cytology personnel compared
to their performance in both glass slide PT and CBPT, by using manual rescreening
to assess work performance. Several committee members then discussed the three
recommendations of the Cytology Subcommittee that were accepted by the CLIAC
in December 1993. The Committee reaffirmed the three recommendations (as
stated in the CLIAC minutes for December 1993) for phased implementation of PT
in cytology:
encourage the development of private or state administered programs that
provide supervised glass slide PT and meet the current regulations
concurrently pursue the legislative and/or regulatory changes necessary to:
1) develop approvable alternative PT programs
2) allow testing to be supervised, but not necessarily performed on-site
3) allow the use of simulations of glass slides, e.g., computer images or
transparencies
promote the development of computer technology that will test both locator
and interpretive skills
PUBLIC COMMENTS
1. Dorothy Rosenthal, M.D., former Chairperson of the CLIAC Cytology
Subcommittee, stated that she is concerned about the outcome of cytology PT,
specifically "the effectiveness of either glass slide or facsimile based PT as a
long term solution for assuring the quality of cytology testing", as stated in
the December 1993 CLIAC minutes. We should consider the money and the
resources that have been expended to implement cytology PT.
2. Mr. Roger Wall, President of Diagnostic Cytology Laboratory, Inc. (DCL),
described the educational program of DCL, Current Education in Cytology
(CEIC). CEIC has a mailed glass slide program which currently serves 180
cytology laboratories in the U.S. and a non-gynecologic glass slide program.
Mr. Wall said that it is necessary to evaluate the most difficult cases, not just
routine cases, in order to improve the quality of cytology and that 10 slides
per year is inadequate for this purpose. He felt strongly that any evaluation
test should include a continuing education program. In addition, he thought
that the potential for nationwide glass slide testing still exists, perhaps
under different conditions than those currently required under the CLIA
regulations.
Committee Discussion
The Chairman commented that CLIAC agrees that "PT is not the whole story"and
that education is essential to improve quality. Another committee member wanted
to discuss changing the regulations based on Dr. Anderson's suggestion. The
Chairman said that some decisions need to be made about CBPT before discussing
regulatory changes, but assured the committee member that Dr. Anderson's
statement would be obtained in writing for future review. Another member
suggested that CDC's Attorney-Advisor should review the statement when it is
received.
CONCLUDING REMARKS
Dr. Schwartz announced that the dates for upcoming CLIAC meetings will be
May 29-30 and September 25-26, 1996. He then adjourned the meeting.
