| March 1996 Meeting Summary: |
|
| Table of Contents |
| I. Record of Attendance |
| II. Welcome and Announcements |
| III. Topics |
|
| CLIA Update |
|
| Centers for Disease Control and Prevention |
|
| Health Care Financing Administration |
|
| Overview of Comments to Cytology Proposed Rule |
|
| Presentations on Automated Instruments for Pap Smears |
|
| AutoPap |
|
| Papnet |
|
|
CDC Cooperative Agreements on Computer-Based Cytology Proficiency Testing
American Society of Clinical Pathologists
Thomas Jefferson University
New England Medical Center |
|
| Solicited Public Comments |
|
| IV. Public Comments |
|
| V. Concluding Remarks |
|
| VI. The Addenda |
|
| Record of Attendance |
|
| The Clinical Laboratory Improvement Advisory Committee (CLIAC) met at the
Centers for Disease Control and Prevention (CDC), Auditorium B, in Atlanta, Georgia, on
March 6, 1996. Those in attendance are listed below: |
|
| Committee Members |
Ex Officio Members |
| Dr. J. Scott Abercrombie |
Dr. Carlyn Collins, CDC |
| Dr. Thomas Bonfiglio |
Dr. Steve Gutman, FDA |
| Ms. Michele Best |
Ms. Judith Yost, HCFA |
| Dr. Ronald Cada |
| Dr. Susanne Gollin |
| Dr. Verlin Janzen |
Executive Secretary |
| Ms. Sandra Johnson |
Dr. Edward Baker |
| Dr. J. Stephen Kroger |
| Dr. Bereneice Madison |
| Dr. Deborah McHugh Liaison Representatives |
| Dr. Wendell O'Neal Dr. Fred Lasky (HIMA) |
| Dr. Glenda Price |
| Dr. Sharon Radford |
| Dr. Patricia Saigo |
| Dr. Morton Schwartz |
| Mr. Elliott Segal |
|
| Centers for Disease Control and Prevention |
|
| Ms. Diane Bosse |
| Ms. Cheryl Coble |
| Ms. Debbie Coker |
| Ms. Carol Cook |
| Ms. MariBeth Gagnon |
| Dr. Richard Keenlyside |
| Mr. Tommy Lee |
| Dr. John C. Ridderhof |
| Mr. Gregory Smothers |
| Dr. Tina Stull |
| Ms. Julie Wasil |
| Ms. Rhonda Whalen |
|
| Clinical Laboratory Improvement Advisory Committee |
|
| The Secretary of Health and Human Services is authorized under Section 353
of the Public Health Service Act, as amended, to establish standards to assure consistent,
accurate, and reliable test results by all clinical laboratories in the United States. The
Secretary is authorized under Section 222 to establish advisory committees. |
|
| The Clinical Laboratory Improvement Advisory Committee (CLIAC) was
chartered in February 1992 to provide scientific and technical advice and guidance to the
Secretary and the Assistant Secretary for Health regarding the need for, and the nature
of, revisions to the standards under which clinical laboratories are regulated; the impact
on medical and laboratory practice of proposed revisions to the standards; and the
modification of the standards to accommodate technological advances. |
|
| The Committee consists of 20 members, including the Chair. Members are
selected by the Secretary from authorities knowledgeable in the fields of microbiology,
immunology, chemistry, hematology, pathology, and representatives of medical technology,
public health, clinical practice, and consumers. In addition, CLIAC includes three ex
officio members, or designees: the Director, Centers for Disease Control and Prevention;
the Commissioner, Food and Drug Administration; the Administrator, Health Care Financing
Administration; and such additional officers of the U.S. Government that the Secretary
deems are necessary for the Committee to effectively carry out its functions. CLIAC will
also include a non-voting liaison representative who is a member of the Health Industry
Manufacturers Association and such other non-voting liaison representatives that the
Secretary deems are necessary for the Committee to effectively carry out its functions. |
|
| Due to the diversity of its membership, CLIAC is at times divided in the
guidance and advice it offers to the Secretary. Even when all CLIAC members agree on a
specific recommendation, the Secretary may not follow their advice due to other overriding
concerns. Thus, while some of the actions recommended by CLIAC may eventually result in
changes to the law, the reader should not infer that all of the advisory committee's
recommendations will be automatically accepted and acted upon by the Secretary. |
|
| Welcome and Announcements |
|
| The meeting was called to order by CLIAC Chairman Dr. Morton Schwartz. The
committee members made self-introductions and disclosure statements of their relevant
financial interests as they relate to any topics to be discussed during the CLIAC meeting.
Dr. Schwartz then stated the role and function of CLIAC. Dr. Edward Baker, Director of the
Public Health Practice Program Office, CDC, and Executive Secretary of the CLIAC welcomed
the committee members. He noted the celebration of CDC's fiftieth anniversary in 1996,
preparation for the Olympics to be held in Atlanta, and the upcoming opening of the CDC
World Learning Center. |
|
| CLIA UPDATE/CDC Addenda A-E |
|
| Dr. Carlyn Collins, Director of the Division of Laboratory Systems, CDC,
referred to instructions distributed to Committee members for Internet access to Federal
Register publications, an updated chronology of CLIA-related Federal Register
publications, and the status of CLIAC recommendations as of March 1, 1996 (see Addendum
A-C). She noted that action on certain CLIAC recommendations is pending the development of
a final, final regulation, which may be published by the end of the year. |
|
| Dr. Collins reviewed the comments and concerns of 43 letters (320
comments) received in response to the publication of the waived proposed rule (see
Addendum D). She stated that, for the past year, CDC has been accepting requests for
waiver based on guidelines in the proposed rule and listed the test systems waived under
these guidelines. Then Dr. Collins reviewed the content of 51 comment letters (304
comments) received in response to the publication of the accurate and precise technology
(APT) proposed rule (see Addendum E). |
|
| Mr. Kevin Malone, Attorney-Advisor, CDC, presented the background for an
appeal by the Department of Health and Human Services (DHHS) to have a court ruling by
U.S. District Judge Gladys Kessler reversed. On two of four points, the judge ruled in
favor of the plaintiffs, Public Citizen and Consumer Federation of America, stating that
(1) the CLIA criteria for test categorization failed to consider the risks and
consequences of erroneous results to patients, and (2) proficiency testing (PT) for
cytologists does not reflect "normal working conditions". DHHS feels that the
current regulation takes risks into account and that test categorization based on the
potential consequences of erroneous results would be too subjective and unworkable. In
addition the Department believes that, to the extent practicable, "normal working
conditions" are incorporated in the cytology PT requirements in the current
regulation. DHHS appealed the court ruling and hopes to have a decision by the end of the
year. As required by the court ruling, DHHS published a cytology proposed rule in November
1995. |
|
| Committee Discussion |
|
| One CLIAC member asked for clarification on the effective date of waiver
for a test system. Dr. Collins responded that waived categorization is effective when the
manufacturer is notified, which precedes the publication of a Federal Register notice
announcing waiver approval. Another committee member asked if any requests for waiver have
been denied. Dr. Collins responded that 44 requests have been received and none have been
denied to date, but not all waiver submissions have been approved. She noted, however,
CDC, in evaluating waiver requests, works closely with the manufacturers. In addition, Dr.
Collins commented that the criteria for waiver approval by CDC are similar, but not
identical, to the criteria for home use approval by the Food and Drug Administration
(FDA). Dr. Schwartz stated that the problem is that, if the manufacturer of a device
approved for home use applies for waiver, CDC must approve the request for waiver. He then
said that he hoped that CDC and the Food and Drug Administration (FDA) were working
together to streamline the waiver approval process. Then Dr. Schwartz asked Dr. Collins if
CLIAC should revisit the guidelines for waiver. Dr. Collins responded that the proposed
clarifications to the criteria for waiver (as included in the CDC guidelines and published
in the waived proposed rule) appear to work well, noting that the guidelines are objective
and easy to apply. |
|
| With respect to APT testing, Dr. Schwartz remarked that CLIAC did not
unanimously support the proposed APT subcategory. Several other committee members noted
the previous CLIAC discussions that the establishment of the proposed APT subcategory
would not provide sufficient regulatory relief to laboratories and should be reconsidered.
Another member asked if the subcommittee on test categorization would have further
opportunity to discuss APT. Dr. Collins replied that we anticipate additional discussions
with the Health Care Financing Administration (HCFA) and within the Department. |
|
| One committee member asked how the result of the appeal process would
affect the content of the cytology final rule. If the ruling of the lower court is
overturned, Mr. Malone suggested that the current regulation would remain in place.
However, if the decision of the lower court is upheld, DHHS will probably reevaluate
options for cytology PT. Mr. Malone noted the inability of the Department to implement
glass slide PT on a national scale, and said that responses to the cytology proposed rule
indicated that computer-based PT (CBPT) might be an alternative testing media. |
|
| CLIA UPDATE/HCFA Addenda F-G |
|
| Ms. Judith Yost of the HCFA presented a status report on CLIA
implementation and a summary of the types of deficiencies sited during surveys(see
Addendum F). She noted consistency between the types of deficiencies cited in first and
second cycle surveys, i.e. failure to enroll in PT continues to be the most frequently
cited deficiency, followed by failure to perform or document quality control (QC)
activities, and absence of a quality assurance (QA) plan. Approximately 25-30% fewer
deficiencies in quality-related areas were cited in second cycle surveys. Ms. Yost
attributed the decrease in deficiencies to HCFA's educational approach in conducting
inspections. Physicians' office laboratories continue to have high numbers of
deficiencies, especially failure to perform and document two levels of QC testing each day
of use and failure to follow the manufacturer's test system instructions. |
|
| As a part of Vice-president Gore's program to reinvent government, HCFA
recently began a performance-based survey process, in lieu of on-site inspection.
Laboratories with good performance on their first cycle survey (approximately 10-15%
excluding cytology laboratories) and satisfactory PT results are eligible for evaluation
by a self-assessment questionnaire (Alternative Quality Assessment Survey) for one
two-year cycle. Random on-site surveys will be conducted of the laboratories receiving a
performance-based survey. All laboratories will be surveyed on-site at least every four
years. |
|
| A revised on-site survey process, which focuses on quality outcomes, is
also being initiated. A surveyor will begin the inspection by reviewing the laboratory's
QA program. If the observed outcomes are good, the laboratories will be in compliance with
CLIA regulations. If problems in QA are detected, the surveyor will more extensively
evaluate the laboratory operation to determine areas of non-compliance. Survey
instructions for the process are currently being developed and training of surveyors will
begin in May. The new process is expected to be more efficient and effective. |
|
| Nineteen proficiency testing programs have been approved for 1996. Oregon
has been approved as a State exempt program. Interest in State exemption has been
expressed by California and Georgia. |
|
| Ms. Yost commented that HCFA has received a number of questions concerning
the applicability of the CLIA regulations to the AutoPap 300 and the Papnet, automated
cytology devices recently approved by the FDA. She reviewed the intended use of each
device and stated that laboratories using these devices will be in compliance with CLIA
regulations if they follow the manufacturers' instructions (see Addendum G). |
|
| Committee Discussion |
|
| Several committee members voiced concerns about HCFA's statistical data.
The following potential problem areas were noted: |
|
| 1. For a given laboratory, the laboratory's type is self-reported and may
be incorrect. |
|
| 2. Inspection data on accredited laboratories (61% of the nation's
laboratories) are not included in the laboratory statistical reports developed by HCFA.
Most hospital and independent laboratories have an accreditation certificate and are not
included in the HCFA database. Accrediting organizations such as the College of American
Pathologists (CAP), the Joint Commission on the Accreditation of Health Organizations, the
Commission on Office Laboratory Accreditation (COLA), and exempt states (New York,
Washington, and Oregon) maintain their own data and information about laboratories that
participate in their programs. |
|
| 3. Data is not up-to-date. |
| 4. Data included in the table of the top 20 deficiencies is cumulative. If
a laboratory was cited for the same deficiency in its first and second cycle surveys, the
deficiency and the laboratory were counted twice. |
|
| 5. No deficiency data is available on laboratories that have a certificate
of waiver. |
|
| 6. Data on enforcement actions for 1995 is not yet available. |
|
| Ms. Yost agreed that there are a number of problems with the data, but
said that she anticipates having better data by the next CLIAC meeting, as well as a list
of laboratories cited for immediate jeopardy. She stated that accreditation organizations
are required to report enforcement actions to HCFA and that each year HCFA performs
validation survey of 3-5% of accredited laboratories; she noted, however, that the
requirements of accreditation organizations must be equivalent but not identical to HCFA
requirements. Dr. Kroger reported that COLA's data also showed a decrease in the number of
deficiencies cited during the second cycle inspections, and the deficiencies cited are
similar to those cited by HCFA. |
|
| In response to concerns expressed earlier by several committee members,
Dr. Rabinowitz confirmed that the waiver and home use approval processes do not mesh
exactly. Several committee members commented that the intended use of a device waived
under CLIA is often quite different from a device approved for home use and were concerned
about the safety and effectiveness of a home use device (such as a glucose monitor) being
used in a non-home setting (such as a nursing home). Dr. Rabinowitz explained that the
FDA's responsibility is to review submissions from the manufacturers based on the
manufacturer's claim, i.e. they review data on the intended use of a device to show its
safety and effectiveness. Dr. Schwartz again commented that the law suggests that the CDC
must waive a home use product if the manufacturer applies for waiver status. Another
committee member suggested that the CDC should require data to substantiate the safety of
a home use device when used in a non-home setting. |
|
|
| OVERVIEW OF COMMENTS TO CYTOLOGY PROPOSED RULE Addendum H |
|
| Ms. Rhonda Whalen summarized the comments received to the proposed rule to
revise the time frame for conducting cytology PT. Of the 757 comment letters received,
almost all were opposed to the proposed rate change. Almost 50% of the letters provided
comments on computer-based proficiency testing (CBPT). It was noted that the numbers may
change since CDC continues to evaluate comments. |
|
| Ms. Whalen stated the most common reasons given for opposition to the rate
change and listed suggested alternatives to the proposed PT time frame. With respect to PT
simulating workplace testing, commenters reiterated the differences mentioned in the
preamble to the proposed rule, and noted that three states require a lower maximum daily
workload than the maximum workload rate in the CLIA regulation. Commenters also expressed
concerns about the feasibility and validity of a national cytology glass slide PT program,
and suggested that the regulations be revised to approve the glass slide programs that are
currently available. Approximately 25% of those who commented on CBPT were opposed, while
the remainder were supportive or apprehensive about such a program. Ms. Whalen discussed
the commenters' concerns and suggestions for implementation of a CBPT in cytology. |
|
| Committee Discussion |
|
| One committee member asked if cytotechnologists or pathologists supported
CBPT as an alternative to glass slide PT. Ms. Whalen replied that the data were not
analyzed in that manner, but that the data analysis is not yet complete. Another member
asked about the number of comments from consumers and Ms. Whalen stated that no commenters
identified themselves as consumers. Still another member asked if the publication of a
final rule will be delayed until the outcome of the appeal process is known. Ms. Whalen
noted that some of the comments to the proposed rule have helped in preparation of a
response to the court order. Dr. Baker responded that the development and publication of a
final rule takes several months, so publication is unlikely to occur prior to a decision
in the appeal process. |
|
|
| PRESENTATIONS ON AUTOMATED INSTRUMENTS FOR PAP SMEARS |
|
| AutoPap Addendum I |
|
| Dr. Tom Anderson first explained the theory and operation of the AutoPap
300 approved by the FDA to assist in laboratory QC in cytology. He then described
components of the AutoPap system and the use of the instrument, noting that the AutoPap
contains a comprehensive quality assurance system integrity device to assure that the
instrument is constantly in calibration. Dr. Anderson reported that, in the clinical
evaluation of 12,000 slides, the instrument detected 34% of the false negatives in all
diagnostic categories and more than 50% of the false negatives in the categories of low
and high grade squamous intraepithelial lesions and cancer, when compared to manual
rescreens of all negative slides. Eventually, the company will request FDA approval of the
instrument for cytology PT and for primary screening of Pap smears. Dr. Anderson asked the
CLIAC to consider revisions to the CLIA regulations appropriate for this type of
technology and proposed specific language to revise the CLIA regulations at
493.1257(d)(1)(I) as follows: |
|
| The review must include negative cases selected by an automated cytology
device approved by FDA for use in quality control or at random from the total caseload and
from patients or groups of patients that are identified as having a high probability of
developing cervical cancer, based on available patient information. |
|
|
| Papnet Addendum J |
|
| Dr. Laurie Mango described the intended use of the Papnet in the clinical
setting. All Pap smears categorized as negative by manual screening would be referred to a
Neuromedical Systems, Inc. regional scanning center. Smears would be rescreened by Papnet
and the most suspicious 128 images from each smear would be digitized. The digital tape
and the original slides would be returned to the laboratory, where the digital images
would be examined at a Papnet review station by cytotechnologists. The original smears
would then be evaluated microscopically as necessary. Papnet thus assists in human
decision making, i.e., it locates potentially abnormal cells, but requires that the
cytotechnologist or pathologist make the final interpretation. |
|
| When the combination of manual screening with Papnet review of slides was
compared to manual screening alone in clinical trials, abnormality was found in over 4.8%
of smears routinely identified as normal at these clinical sites, i.e., a 30% increase in
relative sensitivity. The instrument is highly effective for the detection of low numbers
of abnormal cells. |
|
| Committee Discussion |
|
| Committee members asked the manufacturers for clarification of the
criteria for false negatives in the clinical trials of both instruments. In some instances
in both studies, the criteria for false negative Pap smears was based on clinical
outcomes, i.e. biopsy confirmation of disease. More often, however, that data was not
available. A few committee members were very concerned about the high false negative rates
reported in the studies (20-25%), and agreed that this statistic should be related to
patient outcomes, not just to laboratory results. The CLIAC chairman noted that
"worst case" published data indicated a false negative rate of about 11%.
Another committee member commented that ASCUS (atypical squamous cells of undetermined
significance) is responsible for a large component of the false negatives. |
|
| Several committee members wanted information on the impact of the
instruments on turn-around-times and workflow in the laboratory. Since the AutoPap is
located in the laboratory and QC slides can be loaded on the instrument as soon as manual
screening is completed, there should be little effect on turn-around-times. Laboratories
using the Papnet must ship the slides to a regional center which scans the slides, makes
digital tapes, and ships the slides and tapes back to the laboratory. The turn-around-time
for this is 3-5 working days. In the laboratory, cytologists then examine the digital
images at a review station and select the slides requiring manual review. |
|
| Committee members asked specific questions about the operation of Papnet.
Dr. Mango briefly described the "training" of the computer with normal and
abnormal images, scoring the cells, the reproducibility of the cell selection process, and
noted that high risk false negatives should be screened manually. Several committee
members asked about the cytotechnologists' review of the digitized images. Dr. Mango
responded that a cytologist should not be biased by the order of the images presented
(they are displayed according to geographic location, not according to hierarchy of
abnormality). Although the same number of images is presented from every slide, the
cytotechnologists in the trial studies selected about 25% of the slides in the control
cohort for human review. Therefore, if a laboratory rescreened 100% of its negative slides
with Papnet, this would represent approximately a 15% increase in workload above the 10%
required for random rescreen. Another committee member asked if the manufacturer claimed
that Papnet is more accurate than a 100% manual rescreen. Dr. Mango clarified that, based
on the data from a routine 10% CLIA mandated rescreen, the manufacturer predicted that
Papnet would detect more abnormal smears. The same committee member felt that the
manufacturer should not make this claim unless Papnet results were compared to the results
of a 100% manual rescreen. |
|
| The Health Industry Manufacturers Association (HIMA) liaison noted that
there had been a 5-month delay in HCFA's review of the AutoPap for CLIA compliance. He
commented that problems of this nature can delay laboratory use of the instrument and are
very expensive for the manufacturer. He was in support of the proposal by Dr. Anderson to
change the CLIA regulations to increase flexibility and allow FDA and HCFA to work
together more efficiently. Ms. Yost acknowledged that the furlough of federal workers
contributed to the delay, but stated that in September HCFA and CDC requested additional
information from the manufacturer which was not received until December. |
|
| There was a discussion about the perception of the use of computer-based
images in cytology. One committee member said that the public's major concern about
cytology testing is the false-negative rate. He felt that, although the public might be
skeptical initially about computer-based screening of patient slides, this perception
could be improved through education that the false negative rate could be lowered by using
computer-based screening. Another committee member stated that some of the public already
believe that instruments make fewer errors than humans. Several committee members were
concerned about the negative reactions of cytology personnel to CBPT. One member felt that
the apprehension was understandable, when individuals' jobs are at risk. Another member
noted that people lose their apprehension over time, while another CLIAC member felt that
recently trained personnel would be more receptive to CBPT, since they have been exposed
to computer-based certification examinations. |
|
|
| CDC COOPERATIVE AGREEMENTS ON CBPT IN CYTOLOGY |
|
| Dr. Collins briefly reviewed the requirements for cytology PT. She noted
the Department's inability thus far to implement a nationwide glass slide proficiency
testing program and the pending court case. As a result of discussions with CLIAC in
December 1993 concerning alternatives to glass slides (see CLIAC recommendations in Tab
E), CDC has pursued CBPT as an option to glass slide testing.. Computer-based testing
would provide uniform testing, require fewer glass slides, have options for test
administration, promote equitable testing, and be less expensive. She noted that the major
difficulties encountered are the inability to adequately test locator skills, requirements
for large amounts of computer storage space, inability to test under normal working
conditions, and uncertain acceptance by the professional community. Dr. Collins stated
that CDC has completed three cooperative agreement studies which have explored the
feasibility of CBPT. Ms MariBeth Gagnon of the CDC explained the requirements of the
cooperative agreement and introduced the presenters from the American Society of Clinical
Pathologists, Thomas Jefferson University, and New England Medical Center who presented
results of the pilot studies. |
|
| American Society of Clinical Pathologists (ASCP) Addendum K |
|
| Ms.Theresa M. Somrak, JD, CT(ASCP), referred to the regulatory
requirements for cytology PT and commented that, because of the nature of testing, normal
cytology working conditions cannot be duplicated. However, she emphasized that cytology
workplace skills would be duplicated under testing conditions. Ms. Somrak described the
establishment of a bank of digitized challenges, the development of software to administer
the test, field testing of the images, and pilot testing (including an opinion survey of
the participants) at two scientific meetings. Then Mary E. Lunz, Ph.D., discussed reasons
for using CBPT, including validity and reliability issues. She expressed concerns about
the reliability of a test that includes only ten challenges (as required for glass slide
testing) and suggested that increasing the number of challenges in CBPT would decrease the
error of measurement and therefore increase the precision and reliability of the test. She
then presented the results of the ASCP/CDC pilot study. She said that there was a higher
success rate on the still images than on the scan images of both cytotechnologists and
pathologists; she added that the reliability was low for both the still and scan images,
which she attributed to the small number of challenges. When individuals' performance on
CBPT was compared with their performance on glass slide PT, 73% either passed or failed
both tests, while 27% had discrepant pass/fail results. Dr. Lunz stressed increasing the
reliability of the test and felt that too much emphasis was being placed on the mode of
administration. She then recommended that computer-based images be used in cytology PT.
She said that CBPT could provide a broader range of challenges than glass slide PT, could
adequately test individual competency, should occur in a controlled environment, and
concluded that testing under these conditions would approximate normal working
skills/responsibilities. |
|
| Thomas Jefferson University (TJU) Addendum L |
|
| Ms. Shirley Greening described the design and results of the TJU/CDC
cooperative agreement study. In the pilot study, each individual took a different test.
Attempts were made to simulate normal working conditions, i.e., the ability to examine
each field at low and high powers and to return to previous challenges. Although
participants were allowed 2 hours, most completed the test in 25-30 minutes. Ms. Greening
expressed disappointment in the results of the study. Only 54% of the individuals who
participated in the CBPT passed the test, compared to 80% of those who participated in the
glass slide PT. Many participants commented on the poor quality of the images (vendor
supplied an unsuitable monitor). Some participants who failed had difficulty reading the
text on the screen, said that the selection of cells was inadequate to answer questions,
and felt that the test was not a reflection of their interpretive skills. Most individuals
suggested that the ability to focus up and down on still images would be helpful. Ms.
Greening concluded that the TJU system could currently be used for educational purposes
and is a viable alternative to glass slide PT. She noted that further development of
certain aspects of the TJU system is needed, and that comparison studies would be
necessary to determine valid and reliable performance benchmarks. |
|
| New England Medical Center (NEMC) Addendum M |
|
| Ms. Gagnon apologized to NEMC that CDC was unable to provide compatible
computer equipment for a demonstration. Dr. Martha Hutchinson described the design of the
NEMC system, which includes focus fields and movie loops. Dr. David Zahniser presented the
results of pilot studies performed at the ASCT and ASCP meetings and later at local
laboratories in the Boston area. Using ten CBPT challenges, 17% of the participants failed
the test. For 37 participants in both the CBPT and the glass slide PT, the pass/fail
results were in agreement, while 11 individuals had discrepant results. In general, the
participants liked CBPT as a testing method, were impressed with the quality of the
computer images, and liked the ability to focus. Participants suggested that the clinical
history should remain on the screen, and that improvements should include more and larger
images, more high power fields, strict Bethesda report groupings, and a light setting
option. Dr. Zahniser noted that the failure rates of NEMC and ASCP were similar, and
suggested that a computer-based test should contain 20 slides. He pointed out that the
NEMC system monitored the number and location of clicks made by the participants.
Individuals with less than five years of experience clicked more often on a challenge and
required longer examination time than individuals with more than five years of experience.
With further development of this feature, Dr. Zahniser felt that the NEMC system could
evaluate locator as well as |
| interpretive skills. |
|
| Committee Discussion |
|
| The CLIAC chairman stated that the issues for discussion were (1) the
automated instruments that have been approved by the FDA for cytology QC and (2) the use
of computer-based images in cytology PT. He noted the potential use of the automated
instruments in PT and commented that changing the language of the regulations, as
suggested by Dr. Anderson, would be a long, laborious process . |
|
| A committee member asked about the future direction of CDC's efforts in
the area of cytology PT. Dr. Collins said that the direction is currently undecided.
However, she noted that rule making is in process to revise regulations pertaining to PT
workload rate and stated that CDC had previously considered revising the CLIA regulations
to allow approval of a variety of PT methods. The same committee member thought that
results of the cooperative agreement studies were valuable, and suggested that CDC enhance
"the best of all three" with additional technology, e.g., the use of CD ROMs for
storage. In addition, she suggested that there should be 20-30 challenges, instead of 10.
Another committee member agreed, adding that the number of challenges should be increased
for any PT methodology. |
|
|
| Yet another committee member commented that she felt evaluating only the
locator skills of cytotechnologists is adequate, since the abnormal cells would then be
referred to a pathologist for interpretation. She asked for and received clarification of
the differences in grading the performance of cytotechnologists and pathologists. Dr.
Collins commented that poor locator performance may reflect a lack of attention instead of
a lack of skill. The same committee member asked about possible bias in the selection of
participants in the pilot studies. The presenters commented that anyone who volunteered at
the professional meetings was tested, but acknowledged that the results could be biased
toward individuals who attend regional meetings. |
|
| Another committee member asked about the possibility of taking a CBPT
on-site by simply logging into a central computer if the laboratory has the necessary
computer equipment. One of the presenters responded that, while it may be possible, the
logistics of administering a computerized test are quite complex. She proposed as an
alternative, using existing testing centers across the country, where testing could be
monitored, or suggested simply mailing and returning a disk if one is not concerned about
monitoring. A committee member, who had participated in the ASCP pilot study, commented
that she adapted easily to the testing method, and felt that the perception of
computerized testing would not be an issue. She asked about the possibility of using a
minimum number of slides to detect competent individuals, and an increasing number of
slides (with a maximum limit) to detect incompetent individuals, as is done with the ASCP
Board of Registry. The ASCP presenter felt that this would be an option with CBPT, as
would increasing the standard number of challenges. She said that either option would
increase the reliability of the test. Another committee member, who agreed that adaptation
to computerized testing is not an issue, felt that CBPT is currently the only viable
option to glass slide PT, and that it could definitely improve the reliability of testing.
|
|
| Solicited Public Comments Addenda N-O |
|
| 1. Dr. Mary Nielson read a statement (see Addendum N) reflecting CAP's
views on computer-based programs for cytology PT and recommendations for changes in the
current CLIA regulations, and described the CAP PAP PT program. The committee chairman
asked Dr. Nielson if any of the CLIAC presentations would have affected the CAP position
statement. Dr. Nielson responded that the CAP's position, that glass slides are the best
approach for PT, would not have changed. |
|
| 2. Kathy Grant, Ph.D., representing ASCP, described her research in
computer- based PT in cytology (see Addendum O). Based on the results of her independent
study, she recommended constructing a field-tested bank of challenges using existing
computer technology and increasing the number of challenges to 25-50 to improve the
reliability of the testing method. She noted that her conclusions were similar, but not
identical, to those of the ASCP. |
|
|
| Committee Discussion |
|
| One CLIAC member was concerned that since the three pilot studies are
completed, CDC's involvement in CBPT would end. He recommended that CDC continue to pursue
the development of CBPT and another committee member agreed. Dr. Collins clarified that
although the pilot studies are completed, CDC has a 2-year contract to study the actual
work performance of cytology personnel compared to their performance in both glass slide
PT and CBPT, by using manual rescreening to assess work performance. Several committee
members then discussed the three recommendations of the Cytology Subcommittee that were
accepted by the CLIAC in December 1993. The Committee reaffirmed the three recommendations
(as stated in the CLIAC minutes for December 1993) for phased implementation of PT in
cytology: encourage the development of private or state administered programs that
provide supervised glass slide PT and meet the current regulations concurrently pursue the
legislative and/or regulatory changes necessary to: 1) develop approvable alternative PT
programs 2) allow testing to be supervised, but not necessarily performed on-site 3) allow
the use of simulations of glass slides, e.g., computer images or transparencies promote
the development of computer technology that will test both locator and interpretive skills
|
|
|
| PUBLIC COMMENTS |
|
| 1. Dorothy Rosenthal, M.D., former Chairperson of the CLIAC Cytology
Subcommittee, stated that she is concerned about the outcome of cytology PT, specifically
"the effectiveness of either glass slide or facsimile based PT as a long term
solution for assuring the quality of cytology testing", as stated in the December
1993 CLIAC minutes. We should consider the money and the resources that have been expended
to implement cytology PT. |
|
| 2. Mr. Roger Wall, President of Diagnostic Cytology Laboratory, Inc.
(DCL), described the educational program of DCL, Current Education in Cytology (CEIC).
CEIC has a mailed glass slide program which currently serves 180 cytology laboratories in
the U.S. and a non-gynecologic glass slide program. Mr. Wall said that it is necessary to
evaluate the most difficult cases, not just routine cases, in order to improve the quality
of cytology and that 10 slides per year is inadequate for this purpose. He felt strongly
that any evaluation test should include a continuing education program. In addition, he
thought that the potential for nationwide glass slide testing still exists, perhaps under
different conditions than those currently required under the CLIA regulations. |
|
| Committee Discussion |
|
| The Chairman commented that CLIAC agrees that "PT is not the whole
story"and that education is essential to improve quality. Another committee member
wanted to discuss changing the regulations based on Dr. Anderson's suggestion. The
Chairman said that some decisions need to be made about CBPT before discussing regulatory
changes, but assured the committee member that Dr. Anderson's statement would be obtained
in writing for future review. Another member suggested that CDC's Attorney-Advisor should
review the statement when it is received. |
|
|
| CONCLUDING REMARKS |
|
| Dr. Schwartz announced that the dates for upcoming CLIAC meetings will be |
| May 29-30 and September 25-26, 1996. He then adjourned the meeting. |