 |
|
Motion Analysis as a Tool for the Evaluation of Oculoplastic Surgical SkillEvaluation of Oculoplastic Surgical Skill
George M. Saleh, MRCSEd, MRCOphth;
Vinod Gauba, MRCOphth;
Dawn Sim, MRCOphth;
Daniel Lindfield, MBBS;
Maedeh Borhani, BSc;
Salim Ghoussayni, PhD
Arch Ophthalmol. 2008;126(2):213-216.
ABSTRACT
| |
Objective To evaluate motion analysis as a discriminator of ophthalmic plastic surgical skill between surgeons of varying experience.
Methods Thirty subjects were divided into 3 groups based on surgical experience: novice (< 5 performed procedures; n = 10), intermediate (5-100 procedures; n = 10), and expert (> 100 procedures; n = 10). Detailed 3-dimensional motion data from surgeons performing 2 oculoplastic surgical tasks on a wet laboratory skills board were obtained using the Qualisys motion capture system. The first task was a deep 3-1-1 suture. The second was skin closure with a continuous suture. The main outcome measures were time, overall path length, and total number of movements. Kruskal-Wallis analysis was performed to evaluate statistical significance.
Results Highly significant differences were found during the skin closure task between all groups for mean time (P = .002), overall path length (P = .002), and number of movements (P = .001). For the deep stitch, highly significant differences were also found for time (P < .001), path length (P < .001), and number of movements (P < .001).
Conclusions Motion analysis, using this technology, was able to differentiate between surgeons of varying experience performing oculoplastic tasks, thus demonstrating construct validity. This technique may be useful in the objective quantitative measurement of oculoplastic skill, with potential applications for training and research.
INTRODUCTION
Training in ophthalmic surgery has been traditionally based on the apprenticeship system. With the innovation of video technology in the past few decades, recorded playback analysis with the supervising surgeon has become a popular method of training. However, this technique has a large interobserver variation1 and lacks quantifiable measures with which changes of surgical skill can be monitored over time.
The assessment of surgical technical skill has become more important in recent years2 and it has been shown that there is a significant correlation between objective measures of manual dexterity and surgical skill with the outcome of a procedure.3 Motion analysis, pioneered by Lord Ara Darzi, MD, KBE, is an emerging validated technique of surgical skill evaluation.4-6 To our knowledge, this method has not previously been applied to oculoplastic surgery.
The Qualisys motion capture instrument (Qualisys Medical AB, Gothenburg, Sweden) is a passive optoelectronic kinematic analysis system, most commonly used for the measurement of body motion, and has been validated and extensively used for gait analysis,7 motor control assessments,8 and upper limb function.9 Such technology is used for Mocap (optical motion capture) in entertainment and virtual reality applications. We describe a new adaptation of this technology to ophthalmic plastic surgery and discuss future potential of this technique for surgical skill evaluation.
METHODS
MOTION ANALYSIS TECHNIQUE AND TECHNOLOGY
The Qualisys ProReflex 500 (Qualisys Medical AB) motion analyzer incorporates a multiple camera system arranged in a 360° fashion. Each motion capture camera unit comprises infrared light-emitting diodes around the lens; a low-noise, high-speed sensor; and a built-in microprocessor (Figure, B). The motion capture camera unit uses the reflected infrared light to detect the position of each marker worn by subjects. These markers are coated with retroreflective tape to amplify their brightness as compared with the skin, clothing, and background (Figure, A). A 3-dimensional image of the markers is then formed by the Qualisys Track Manager software (Figure, C, D, and E). The motion data were captured at a sampling rate of 100 Hz, which is sufficiently high for the movements seen during ophthalmic microsurgery. Simultaneous video recordings were carried out using a digital camcorder, allowing motion tracking data to be synchronized and simultaneously processed with the video recording. Data filtering was performed using a zero-lag, second-order, digital Butterworth filter with a cutoff frequency of 10 Hz to eliminate high-frequency noise.
|
|
|
|
Figure. Surgical skills board and the Qualisys motion capture instrument and software (Qualisys Medical AB, Gothenburg, Sweden). A, The Royal College of Ophthalmologists (London, England) surgical skills board in use, illustrating retroreflective skin markers. The skills board comprises several modules. The artificial skin with the preformed incision (used for the skin suture) can be seen at the back. The metallic hook with the surrounding plastic cylinder (used for the deep suture) can be seen on the right. B, Infrared motion capture camera system. C, Qualisys Track Manager software screenshot showing simulated hand movements. D, Qualisys Track Manager software showing tracking of reflective markers and their respective vector paths. E, Qualisys Track Manager software showing the surgical study setup with relative positions of both the motion capture units and markers.
|
|
|
Following extensive pilot laboratory work calibrating the instrument for ophthalmic use, a distribution that most sensitively and accurately detected motion in the oculoplastic surgical setting was determined. Four 10-mm–diameter retroreflective markers were attached using double-sided adhesive tape to each hand at the second metacarpal head, fifth metacarpal head, midpoint between the base of the second and third metacarpal base, and dorsally on the midpoint of the middle phalanx of the second phalange (Figure, A). The main outcome measures were time, overall path length, and total number of movements because these have been previously validated.6
PARTICIPANTS
Thirty subjects were divided into 3 groups (n = 10 each) based on surgical experience: novice (< 5 performed procedures), intermediate (5-100 performed procedures), and expert (> 100 performed procedures). All subjects were given standardized instruction prior to the tasks, and an independent expert observer was present to ensure correct task completion in all cases.
SIMULATED SURGICAL TASKS
The first task was the insertion of a deep 3-1-1 suture using 6-0 polyglactin 910 (Vicryl; Ethicon, Somerville, New Jersey). It had to be placed around a metallic hook surrounded by a plastic cylinder (Figure, A). The suture needle was passed through the hook and then tied to the metallic frame (rim) with 3 throws on the first knot with 2 subsequent single throws to lock it. The loose ends of the suture were cut. The location of this deep stitch made the manipulation of the instruments more challenging. The second task involved the insertion of a continuous skin suture using 6-0 polypropylene (Prolene; Ethicon) in a preformed skin wound (Figure, A). The task commenced with a 3-1-1 knot placed beyond the skin wound to anchor the suture. The needle was then passed subcutaneously and regrasped inside the wound edges after which 3 subcutaneous bites were performed bringing the skin edges together. The needle was then passed from inside the wound edges and brought up through the skin (parallel to the wound), allowing a final 3-1-1 knot to be applied to secure the distal end of the continuous skin closure task.
For both tasks, a standardized wet laboratory environment was used for all surgeons undergoing testing, with the same instruments, surgical skills board (Royal College of Ophthalmologists, London, England) (Figure, A), and unmounted sutures being provided to all subjects. Subjects were allowed time to familiarize themselves with the environment, but once testing commenced, they were required to complete each task a single time without stopping or restarting.
Statistical analysis was performed using the Kruskal-Wallis test on SPSS (version 14; SPSS Inc, Chicago, Illinois). Statistical significance was set at 0.05. A nonparametric test was chosen because of the sample sizes in each cohort.
RESULTS
Summaries of path length, number of movements, and time are presented in the Table. The results demonstrate significant differences in path length, number of movements, and time taken to complete both surgical tasks, with more experienced surgeons demonstrating greater efficiency in completing the given tasks.
|
|
|
|
Table. Summary of Results for Deep Suture and Skin Closure Tasks
|
|
|
Highly statistically significant differences were found between the 3 grades of surgeons for both tasks. For the placement of the deep suture (task 1) (Table), the greater the degree of experience, the shorter the path length (Kruskal-Wallis, P < .001), the lesser the number of hand movements (Kruskal-Wallis, P < .001), and the shorter the time taken (Kruskal-Wallis, P < .001). For the placement of the subcutaneous skin closure (task 2) (Table), the greater the degree of experience, the shorter the path length (Kruskal-Wallis, P = .002), the lesser the number of hand movements (Kruskal-Wallis, P = .001), and the shorter the time taken (Kruskal-Wallis, P = .002).
COMMENT
The recent drive to develop more objective and standardized systems of evaluation for surgical trainees has been propagated by current changes in both the content and delivery of medical education.10 Techniques that have evolved in ophthalmology so far, though useful, have retained an assessor-dependent element of subjectivity. Kinematic and motion analysis is a purely objective tool in both its acquisition and analysis of data.3, 9, 11-15 The data presented in this study show the ability of motion analysis to objectively discriminate between surgeons of differing levels of experience. For both oculoplastic tasks assessed, the more experienced the surgeon, the more efficiently they completed the tasks (with shorter path lengths, fewer hand movements, and less time). These results show construct validity, the ability to discriminate between surgeons of different experience, when applied to these oculoplastic tasks.
As shown in the Table, the spread, range, and standard deviation of the parameters narrow as the skill level progresses. The results suggest that with greater experience there is a conflation of surgical skills and that this effect is more pronounced with the more complicated the task (skin closure).
Poor clinical outcomes can result from inadequate technical skill, but despite this, there have been few formal attempts to evaluate it.6, 12, 14, 16-17 There has recently been a drive toward an expanded set of tools for surgical evaluation beyond those currently tested in written, clinical, and oral examinations. The Accreditation Council of Graduate Medical Education has endorsed the inclusion of surgical competence as one of the competencies for ophthalmology residents.18 Similar changes are taking place in Europe. Motion analysis may have a role in helping formulate an objective and standardized system for appraising technical skill.19-21
The surgical tasks selected evaluated specific components of technical competence in a wet laboratory environment and thus have inherent limitations. These tasks, however, were selected to represent core oculoplastic skills that residents should be familiar with and to be of different complexity. This tool has the potential for providing structured objective feedback on surgical performance that may be used to monitor progress and target further tuition and thus be a useful adjunct to current systems of evaluation.
Further research on the practical implementation of this method is required, including its potential to evaluate live surgery. Motion analysis, using a different technology, has been successfully applied to corneal suturing under the microscope.6 It is encouraging that this form of motion tracking technology, which to our knowledge has not been used for surgical evaluation previously and is more sensitive than other similar tools, was successfully adapted to oculoplastic surgery. You can escape bad teaching but not bad assessment. Good assessment procedures are fundamental for promotion, certification, and licensure. No single method can comprehensively assess the surgical skills of residents in training. Our results offer encouragement that as further research takes place in this field, motion analysis will prove to be a useful modality in accomplishing the current goal of more objective surgical evaluation.
AUTHOR INFORMATION
Correspondence: George M. Saleh, MRCSEd, MRCOphth, Moorfields Eye Hospital, 162 City Rd, London EC1V 2PV, England (drgmsaleh{at}yahoo.co.uk).
Submitted for Publication: April 1, 2007; final revision received May 21, 2007; accepted June 22, 2007.
Financial Disclosure: None reported.
Author Affiliations: Frimley Park Hospital (Drs Saleh, Sim, and Lindfield) and Department of Biomedical Engineering, University of Surrey (Dr Ghoussayni and Ms Borhani), Surrey, Moorfields Eye Hospital, London (Drs Saleh and Sim), and St James Hospital, Leeds (Dr Gauba), England.
REFERENCES
| |
1. Barnes RW. Surgical handicraft: teaching and learning surgical skills. Am J Surg. 1987;153(5):422-427.
FULL TEXT
|
ISI
| PUBMED
2. Darzi A, Taffinder N. Assessing operative skill: needs to become more objective. BMJ. 1999;318(7188):887-888.
FREE FULL TEXT
3. Datta V, Mandalia M, Mackay S, Chang A, Cheshire N, Darzi A. Relationship between skill and outcome in the laboratory-based model. Surgery. 2002;131(3):318-323.
FULL TEXT
|
ISI
| PUBMED
4. Bann SD, Khan MS, Darzi AW. Measurement of surgical dexterity using motion analysis of simple bench tasks. World J Surg. 2003;27(4):390-394.
FULL TEXT
|
ISI
| PUBMED
5. McBeth PB, Hodgson AJ, Nagy AG, Qayumi K. Quantitative methodology of evaluating surgeon performance in laparoscopic surgery. Stud Health Technol Inform. 2002;85:280-286.
PUBMED
6. Saleh GM, Voyatzis Y, Hance J, Ratnasothy J, Darzi A. Evaluating surgical dexterity during corneal suturing. Arch Ophthalmol. 2006;124(9):1263-1266. [published correction appear in Arch Ophthalmol. 2006;124(12):1801].
FREE FULL TEXT
7. Whittle MW. Gait Analysis: An Introduction. 3rd ed. Oxford, England: Butterworth-Heinemann; 2002.8. Perry J. Gait Analysis: Normal and Pathological Function. Thorofare, NJ: Slack; 1992.9. Hingtgen B, McGuire JR, Wang M, Harris GF. An upper extremity kinematic model for evaluation of hemiparetic stroke. J Biomech. 2006;39(4):681-688.
FULL TEXT
|
ISI
| PUBMED
10. O’Day DM. Assessing surgical competence in ophthalmology training programs. Arch Ophthalmol. 2007;125(3):395-396.
FREE FULL TEXT
11. Martin JA, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84(2):273-278.
FULL TEXT
|
ISI
| PUBMED
12. Reznick R, MacRae H, Martin J, McCulloch W. Testing technical skill via an innovative "bench station" examination. Am J Surg. 1997;173(3):226-230.
FULL TEXT
|
ISI
| PUBMED
13. Cremers SL, Ferrufino-Ponce ZK, Henderson BA. Objective Assessment of Skills in Intraocular Surgery (OASIS). Ophthalmology. 2005;112(7):1236-1241.
FULL TEXT
|
ISI
| PUBMED
14. Cremers SL, Ferrufino-Ponce ZK. Global Rating Assessment of Skills in Intraocular Surgery (GRASIS). Ophthalmology. 2005;112(10):1655-1660.
FULL TEXT
|
ISI
| PUBMED
15. Mills RP. American Board of Ophthalmology Program Directors' Task Force on Competencies: report of the American Board of Ophthalmology Task Force on the Competencies. Ophthalmology. 2004;111(7):1267-1268.
ISI
| PUBMED
16. Anastakis DJ, Regehr G, Reznick RK. Assessment of technical skills transfer from the bench-training model to the human model. Am J Surg. 1999;177(2):167-170.
FULL TEXT
|
ISI
| PUBMED
17. Saleh GM, Gauda V, Mitra A, Litwin AS, Chung AK, Benjamin L. Objective structured assessment of cataract surgical skill. Arch Ophthalmol. 2007;125(3):363-366.
FREE FULL TEXT
18. Accreditation Council for Graduate Medical Education Web site. www.acgme.org. Accessed January 2007.19. Spencer F. Teaching and measuring surgical techniques: the technical evaluation of competence. Bull Am Coll Surg. 1978;63(3):9-12.
20. Annett J. Acquisition of skill. Br Med Bull. 1971;27(3):266-271.
FREE FULL TEXT
21. Wade MG. Developmental motor learning. Exerc Sport Sci Rev. 1976;4:375-394.
PUBMED
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter
What's this?
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
Kinematic Analysis of Surgical Dexterity in Intraocular Surgery
Saleh et al.
Arch Ophthalmol 2009;127:758-762.
ABSTRACT
| FULL TEXT
|
