Abstract
BACKGROUND: Although few studies have been performed recently, several have suggested that some practitioners are not well able to detect preset anesthesia machine faults.
METHODS: We performed a prospective study to determine whether there is a correlation between duration of anesthesia practice and the ability to detect anesthesia machine faults. Our hypothesis was that more anesthesia practice would increase the ability to detect anesthesia machine faults. This study was performed during a nationally attended anesthesia meeting held at a large academic medical center, where 87 anesthesia providers were observed performing anesthesia machine checkouts. The participants were asked to individually check out an anesthesia machine with an unspecified number of preset faults. The primary outcome measures were the written listing of faults detected during an anesthesia machine checkout.
RESULTS: Of the five faults preset into the test machine, participants with 0–2 yr experience detected a mean of 3.7 faults, participants with 2–7 yr experience detected a mean of 3.6 faults, and participants with more than 7 yr experience detected a mean of 2.3 faults (
P < 0.001).
CONCLUSIONS: Our prospective study demonstrated that anesthesia machine checkout continues to be a problem.
IMPLICATIONS: Anesthesiologists attempted to detect faults during anesthesia machine checkout. Participants with more than 7 yr of practice since training detected less than half.
In studies that address the issue of anesthetic mishaps, human error and insufficient preanesthetic checking of the anesthesia machine are a recurring theme. Craig and Wilson (
1) also found that human error was responsible for 65% of the incidents “with failure to perform a preanesthetic check, the most common associated factor.” Fasting and Gisvold (
2) noted that 31% of equipment problems involved the anesthesia machine and breathing circuit, with the main cause of human error being insufficient checking of the anesthesia machine before use, especially between cases. They stated, “In our study, human error was the main contributing factor in one-quarter of cases, and most of these involved the anesthesia machine. The main cause was insufficient checking of the anesthesia machine before use, especially between cases.” The possibility for error and cause for concern regarding anesthesia machine checks are very clear.
In 1984 study (
3), 190 people attending an anesthesia meeting were given 10 min to detect five created faults in a standard anesthesia machine. The average number of faults that were detected was 2.2. Professional background did not influence the score, although the ability increased with those practitioners with more than 10 yr experience. In an effort to reduce or eliminate anesthetic mishaps related to anesthesia machine problems, preoperative checklists have been developed to assure proper functioning of equipment. The Food and Drug Administration released a checklist in 1986 which it revised in 1992 (
4). Professional organizations and anesthesia machine manufacturers have also developed such checklists.
Although few studies have been performed recently, several have suggested that some practitioners are not well able to detect preset anesthesia machine faults (
5–8). About 2% of the closed claims have resulted from gas delivery equipment, with death and permanent damage in 76% of the outcomes (
9). A recent case–control study of anesthesia management characteristics on severe morbidity and mortality demonstrated equipment check with protocol and checklist (odds ratio 0.64), and documentation of the equipment check (odds ratio 0.61) was significantly associated with decreased risk (
10). This study was undertaken to see if there has been any improvement in the ability of practitioners to detect preset anesthesia machine faults and if duration of practice is related to the ability to detect such faults.
METHODS
After IRB approval and informed consent, volunteers were asked to perform a preanesthetic checkout on a provided anesthesia machine, a Ohmeda Excel 210 SE. The volunteers were attending an anesthesia meeting that drew attendance from throughout the United States. All study participants used the same machine, and this machine was preset with five faults. The five faults were chosen from faults seen in actual clinical practice and consisted of a leak in the water trap, an empty oxygen cylinder, a sticky exhalation valve, a dead backup battery, and removal of the oxygen/nitrous oxide fail-safe linkage. A source of electricity and pressurized gas was supplied to the test machine.
Volunteers were observed individually during the study, were not required to correct any detected faults, and were allowed a maximum of 10 min to complete the checkout of the anesthesia machine. Participants were asked to list faults on a data collection tool and to, provide demographic information including length of practice, practice setting, education level attained, and place of training. Study participants were asked to keep their findings to themselves and not share this information with potential future study subjects.
The association between years of practice (0–2, 2–7, >7 yr) and number of faults detected (0–5) was of primary importance. For this analysis, the number of faults detected was compared across experience groups using the Kruskal–Wallis test. To supplement this analysis, pairwise comparisons of the three experience groups were also performed. In all cases, a result was deemed statistically significant when two-tailed tests yielded
P < 0.01 (i.e., α = 0.01). Potential confounds such as highest level of education held, type of practice (academic or other), place of training, and number of operating rooms in principal caregiving setting were analyzed and not found to be statistically significant.
Based on the data from Buffington et al. (
3), the similarity of means among groups was estimated to be within 1.0 and the standard deviation to be within ±1.2. That said, a sample size of 24 in each of the three experience groups was thought to have 80% power to detect a difference in means of 1.0 assuming that the common standard deviation is 1.2 using a two group
t-test with a 0.050 two-sided significance level. The 24 participants per group were based on a normal distribution or parametric format. Previous data suggest that the data may be nonparametric in format. The estimated nonparametric format increased the number of participants necessary per group by 10%. Therefore, 27 participants were thought to be required per group to obtain an 80% power with a nonparametric data format.
RESULTS
We enrolled 87 volunteers in the study. There were 29 participants who had 0–2 yr experience, 23 who had between 2 and 7 yr experience, and 35 participants who had more than 7 yr of experience. The type of participant practice was academic 55% and private practice 45%. The percentage of participants who answered the number of operating rooms at the institution where they practiced was: 75% for >30 operating rooms, 10% for 16–30 operating rooms, 14% for 1–15 operating rooms, and 1% for 0 operating rooms. There were no statistically significant differences among groups in fault detection based upon type of practice or number of operating rooms at the participant’s institution.
The average number of faults found by all study participants was 3.1. The number of faults detected differed significantly (
P < 0.001, Kruskal–Wallis test) across experience groups, with participants with more than 7 yr of experience detecting the fewest faults (
P < 0.001 for pairwise comparison with each of other two experience groups). Practitioners with 0–2 yr experience found a mean of 3.7 faults, participants with 2–7 yr found a mean of 3.6 faults, and participants with more than 7 yr experience detected a mean of 2.3 faults (
Fig. 1). Overall, 74.7% of subjects detected the leak in the water trap and the empty oxygen cylinder, while only 50.6% and 49.4% found the dead battery and oxygen/nitrous fail-safe linkage disconnect, respectively. Ten participants successfully found all five faults; three subjects detected zero faults.

View larger version:
Figure 1. The percentage of participants who found each of the five faults preset into the test anesthesia machine.
DISCUSSION
Our study demonstrates that we continue to have problems detecting anesthesia machine faults despite the publication of multiple checklists. Our study is similar in design and results to that published by Buffington et al. (
3) in 1984. They modified an anesthesia machine to create five faults and exhibited it at a New York State Society of Anesthesiologists meeting. Those who passed the exhibit were asked to examine the machine and see how many of the faults they could identify. The average number of faults which participants were able to identify was only 2.2. They also demonstrated improved detection of anesthesia machine faults in those participants with more than 10 yr of experience. Our results demonstrated statistically worse results from those with more than 7 yr experience.
In another study published in 1991 (
6), 188 anesthesiologists were asked to detect as many prearranged machine faults as possible using their own checkout methods compared to the Food and Drug Administration checklist. None of the participants in their study found all of the machine faults, regardless of the method used. An observational study of participants during a simulated anesthesia session found no difference between university and community anesthesiologists and anesthesia residents in anesthesia machine checkout, although they only checked less than half of the 20 items on the checklist before anesthesia induction (
11). They found no correlation between participant age or years of experience with the number of items checked before induction of anesthesia. Blike and Biddle (
12) developed a new interactive electronic checklist for anesthesia machine checkout. They compared this checklist to the standard checklist and found that the use of both checklists was associated with a large proportion of difficult faults being missed. Olympio et al. (
7) hypothesized that the poor rate of fault detection was because of the lack of adequate training in checkout procedures. They completed a study to determine whether intensive training sessions led to improved anesthesia machine checks. Checkout procedures were improved in this study; however, 11% of their checkout criteria were not even attempted by study participants. The element of human error cannot be discounted. Cooper et al. (
5) found that 70% of the critical incidents noted were attributed to human error.
Our determination of poorer fault detection in the more experienced participants is unusual, and indicates the need for continued education of anesthesia personnel. As noted above, Buffington et al. (
3) found improved detection of anesthesia machine faults in those participants with more than 10 yr of experience. In contrast, Armstrong-Brown et al. (
11) found no difference in years of experience with the number of items checked prior to induction of anesthesia.
There are several limitations of our study. The study was performed predominantly at an anesthesia meeting site; therefore, the checkout was performed at a site remote in place and time from actual anesthesia machine checkouts. We also did not provide a checklist for participants to follow, neither did we necessarily provide the participant with an anesthesia machine they used daily, nor did we provide familiarization with the provided machine before the test was conducted. It is possible that participants may have talked with a previous participant, which may have increased fault detection. Also, there may have been a presumption that there were faults on the part of the study subjects, which may have increased their vigilance for detecting faults. A better test design might have been to randomly present machines with and without preexisting faults to determine whether subjects identified nonexistent faults.
Supported by Mayo Foundation for Medical Education and Research.