Web Based Technology and Continuing Medical Education

Web Based Technology and Continuing Medical Education This dissertation explores the use of Web based technology to enhance and maintain procedural skills in the context of continuing medical education. The research was initiated by the perceived need for novel and alternative methods of providing procedural skills training to health practitioners. This observation, supported by learning needs analysis, led to the design and implementation of a Web based educational resource aimed at doctors and other healthcare practitioners. The assessment part of the research focused on an empirical evaluation of the effectiveness of this Web based educational resource. This dissertation draws on a number of strands of Health Informatics: Principals of Heath Informatics Research Methods Clinical Information Systems Creating Online Educational Resources Whilst on a small scale, the results are relevant to medical educators involved in developing and evaluating web based educational resources. BACKGROUND Medical practitioners receive comprehensive procedural skills training and it is expected that this is maintained and regularly updated to limit skills decay and ensure clinical competency. Skills decay is defined as the loss of a trained or an acquired skill after a period of nonuse. Skills decay rapidly as the period of nonuse lengthens; and the extent of the decay is influenced by the characteristics of the skill and how and when these skills were learnt. Arthur et al., (1998) Skills are classified either as closed loop or open loop tasks. Arthur et al., (1998). Closed loop tasks are fixed sequence tasks with a defined beginning and end, for example, the preoperative anaesthetic machine check. Open loop tasks are tracking and problem solving tasks, for example, managing patients hypoxia. Arthur et al., (1998) in their review on the rate of skills decay and its influencing factors concluded that closed loop tasks decay more slowly than opened loop tasks. Arthur et al., (1998) also mentions that mental tasks decay more quickly than physical tasks and after 28 to 90 days of nonuse of the trained skills, task performance declines by 23% and by 40% after a year of nonuse. Clinical practice alone may be insufficient to prevent skills decay as indicated in a simulated airway management training study Kovacs et al., (2000). Skills decay quickly without practice; and procedural skills are only optimally retained when trainees regularly practiced the procedure on their own, in their own time and received periodic feedback. Training on simulation modalities, mannequins, fresh cadavers and live patients have the potential to successfully teach the procedural skill with significantly less skills decay over time as compared to didactic teaching alone. TI L et al., (2006). The traditional one to one apprenticeship model of medical procedural skills training and the in-hospital continuous medical education and maintenance of a skills base are often inefficient, expensive, and labour intensive. Patients, who are often used as practice tests subjects during skills training, safety is reliant on the medical practitioner skills retention and task competency. Maintaining procedural skills competency may prove to be increasingly more challenging as expense, time constraints, available manpower, lack of resources and patients reluctance to be used as experimental models make this endeavor increasingly impossible to set up. The resultant worldwide move towards competency based training programs and self directed problem oriented based learning has made necessary the search for alternative valid and reliable educational methods for skills training and its maintenance. Fortunately, the last decade has seen an explosion in the use of technology to enhance medical education. Web-based educational programs, computer aided virtual reality situations, and high fidelity simulation has played an increasingly important role in medical education owning to its efficiency, ability to provide flexible learning experiences, multimedia capabilities, and economies of scale and power to distribute instructional content internationally. Vozenilik et al., (2004) In the last 5 to 10 years extensive empirical research has been conducted on the use of computer aided and web-based instruction in medical education where there has been overwhelming support for these mediums of instruction. Unfortunately the literature is strikingly sparse on the use of Web based instruction for procedural skills training and in the few studies where empirical research has been carried out; study designs were not robust enough to withstand interrogation or had inconclusive results. LITERATURE REVIEW A review of the literature was conducted to ascertain what work had been done in the field of Web based learning, medical education and procedural skills training. A CINAHL and Medline search was carried out exploring all citations up to June 2010. The search using Medical Subjects Headings (MeSH) Computer Aided Instruction, Internet, CME returned 322 publications. Adding the MeSH term Review returned 21 reviews of which four were relevant. Replacing CME with Procedural Skills Training produced only one noteworthy empirical research paper and 2 publications worthy of discussion. Relevant systematic reviews of the literature are summarized in the table below: 3 Title Author / Date Findings Review Conclusions Assessment of the Review Internet-Based Learning in Health Professionals: A Meta-analysis Cook et al., 2008 201 eligible studies with qualitative or comparative studies of Internet based learning accounting for 56 publications Internet formats were equivalent to non-Internet formats in terms of learner satisfaction and changes in knowledge, skills and behavior. Internet based learning is educationally beneficial. Comprehensive work with a robust study design. Skills outcomes included communication with patients, critical appraisal, medication dosing, cardiopulmonary resuscitation, and lumbar puncture. Unfortunately the study had many limitations as many publications were poorly designed with low methodological quality, without validity and reliability evidence for assessment scores and with widely varying interventions What the meta analysis did suggest was that no further studies comparing Internet based interventions with traditional methods or no intervention were merited as these types of studies would almost invariably be in favour of Internet Based interventions. The author of the review suggested that the questions that warranted further research would be when and should Internet based learning be used and how could it be effectively implemented giving impetus to the exploration of Internet based skills training and maintenance. Title Author / Date Findings Review Conclusions Assessment of the Review The Effectiveness of Computer-Aided (CAL) Self-Instructional Programs in Dental Education: Rosenberg et al. 2003 1024 articles systematically reviewed. 12 publications included in the final review. Five studies significantly favored CAL. CAL is as effective as other methods of teaching and can be used as an adjunct to traditional education or as a means of self-instruction. This study is a comprehensive review of controlled randomized studies with clear and relevant inclusion criteria assessed with good inter and intra rated reliability. The reviewer limited the study to dental students. Forms of Computer Aided Instruction was not clearly defined or specified in the inclusion criteria. It is unclear whether web based studies were included. The skills referred to in the studies were dental diagnostic not procedural skills. The apparent dearth of studies assessing procedural skills justified the investigation undertaken by this dissertation. Title Author / Date Findings Review Conclusions Assessment of the Review Internet-based medical education: a realist review of what works, for whom and in what circumstances. Wong et al. 2010 249 papers met their inclusion criteria. Learners were more likely to accept a course if it offered a perceived advantage over available internet alternatives, if it was easy to use technically, had elements of interactivity and gave formative feedback. This study is a realist review and the methodology used answered the question of the study which aimed to provide a theory driven criteria to guide development and evaluation of Internet based tools. The findings and guidelines suggested in this review would later be incorporated in the design of the resource to be investigated in this dissertation. Title Author / Date Findings Review Conclusions Assessment of the Review eLearning: a review of Internet-based continuing medical education (CME). Wuton et al. 2004 16 studies met their eligibility criteria Internet based CME programs were as effective as traditional formats of CME A comprehensive and appropriate search of databases. Randomized controlled trials of Internet based education in practicing health care professionals. These results showed that Internet based interventions do have a place in CME and that these effects on skills behavior warrants further investigation. Title Author / Date Review Conclusions Assessment of the publication Procedures can be learned on the Web: a randomized study of ultrasound-guided vascular access training. Chenkin et al. 2008 Web based tutorial may be an useful alternative to didactic teaching for learning of procedural skills A randomized control trial with non inferiority data analysis. The non inferiority margin was specified at a 10% margin however the actual amount of improvement was not specified. Blinding bias was not assured and the trial relied on the reputation of the investigator. No mention of inter rated reliability was made. Despite its inherent weaknesses, the trial suggested that web based intervention is as good as the alternatives; however, the study incorporated the use of simulation and live models to teach the actual procedural skill. David Cook is a prolific writer of many reviews and publications investigating Internet based formats in medical education. His noteworthy publications Web based learning: pros, cons and controversies Cook, (2007) and Where are we with Web based education Cook,( 2006) extolled the benefits overcoming barriers of distance and time with novel instructional methods, and extenuated the disadvantages which included social isolation, upfront costs and technical difficulties of Web based education. He concluded that Web based instruction can be a potentially powerful tool and strongly recommended that the focus of future studies should concentrate on the timing and application of Web based learning tools. Summary of literature survey The review of the literature has outlined the use of Web based procedural skills training as an area that requires further research. Empirical research and systematic reviews that has been carried out thus far has been limited. The literature research conducted for this dissertation (though in its self may have been limited) was unable to find publications exploring the whole use of the Internet as a means of procedural skills training and skills maintenance. Justification and Learning needs analysis To assess the effectiveness of an Internet based learning resource in the context of procedural skills training, a skill had to be chosen that was relevant, involved both a physical and mental task, and had the potential of decaying. A procedural skill is defined as the mental (knowledge) and motor activities (behaviour) required to execute a manual task and usually involves patient contact. Kovacs (1997). Furthermore, a learning needs analysis was undertaken to assess the value of this topic choice. Justification Intubation with a Laryngeal Airway Device (LAD) was chosen as the representative procedural skill. When a patient collapses from a cardiac or respiratory cause, timely control of the patients breathing and airway with prompt delivery of cardiopulmonary resuscitation (CPR) and defibrillation have resulted in life saving survival and neurological recovery. The LAD is a breathing maintenance device that can, with minimal training, be inserted effortlessly into the mouth of the patient allowing for breathing and oxygenation. It is increasingly being used in the repertoire of techniques available to frontline practitioners (practitioners first on call to resuscitation events) in emergencies where the technique has proven to be easy to use and life saving in the management of an airway crisis. Kette, (2005). In a survey of family medicine practitioners, all practitioners surveyed agreed that insertion of an LAD during a resuscitation procedure was a core procedural skill that most practiti oners were required to perform in any setting; 86% admitting that they had been called upon at some point to perform the procedure. Wetmore et al., (2005). Insertion of a LAD with knowledge of the patients anatomy, indication and contraindication for use and technique of use both under a controlled setting and in an emergency is representative of a procedural skill a frontline practitioners is expected to perform. Learning needs analysis A key step in developing an effective educational website is performing a learning needs analysis to determine what the learning needs the resource hopes to address are and why these needs were not met by existing learning or teaching arrangements. Cook Dupras, ( 2004). A questionnaire not previously validated, making use of closed type questions, were used to assess three broad areas; knowledge and training, skills application and Internet accessibility. Eleven frontline practitioners were asked to provide an indication of how often they were called to attend resuscitation or airway management situations in the last two years. They were surveyed regarding training received in LAD usage and insertion, their desire to obtain more information or skills updating, and whether there was a perceived need for Internet-based continuing medical education courses on LAD usage and intubation. Their attendances at CME workshops in the last year were surveyed and the barriers to CME workshop attendance were assessed. The respondents were surveyed regarding access to the Internet and previous exposure to e-learning modules. This was done to assess whether the uptake of the resource would be biased towards participants with Internet access, frequent Internet uses or previous e-learning experiences. The results of the learning needs analysis showed that most respondents (90%) received exposure to the device. It is a requirement of their post as frontline practitioners, to be Acute Life Support (ALS) trained where usage of the device in resuscitation is taught. Half the respondents indicated that they were not comfortable with their level of knowledge; and 63% felt unconfident about inserting the device as they were on average, only exposed to two resuscitation scenarios per year. All had Internet access at work and at home; and half had previous experience of online learning. Only one percent of the respondents were able to attend a CME session in the last year, citing lack of time and convenience as the main reasons. 80% of respondents were interested in taking courses through the Internet, as continuous education credits are a requirement of a license to practice in medicine. Interest in the topic was high and given the above self-appraisal, it was felt that the course was nee ded and should appeal to this population. METHODS The method section is dealt with in two parts. The first will focus on the development and design of a Web based educational resource and the second on the evaluation of the resource. 1. Development and design The idea was to develop an educational resource that could be used to train, reinforce knowledge and maintain a procedural skill by employing and integrating principles of effective adult learning with the unique features of the web. The development was driven by educational needs and outcomes of learning needs analysis completed by participants in a previous part of this study. 1.1 Development Theories The course design reflected Adult Learning Principles and the aim of the course was to improve knowledge (cognition), integration of attitude changes (confidence) and in so doing result in a change in behaviour (competency). Gale (1986). With accessibility of the Web based educational resource, it was hoped that the resource would be accessed frequently until the task becomes automatic or accessed as a refresher when required or at regular intervals. The resource incorporated principals that were shown to be effective. It was centered on the learners needs, was focused on a specific task and recognised past experiences of the learner (Gale 1986). The theories used in the development of the resource included; Experiential Learning Theory, which concluded that experiential learning should have personal relevance, should be self-initiated and lead to pervasive effects on the learner. Rodgers (1969) Constructivist Theory where learning is an active process with learners constructing new ideas and concepts based upon past and current knowledge. Bruner (1966) Information Processing Theory where knowledge is presentation in sequences or chunking to accommodate short attention spans (Miller 1956). The educational resource strived to be pedagogically sound uniquely applying these principles online. Information was presented in small chunks in a sequential fashion, was self-contained, had interactive components and contained assessments with instant feedback. Online communication did not occur in real time as which happens with video conferencing and online chat rooms, instead the resource used communication that was asynchronous where participants logged on, viewed and downloaded course material, read postings and submitted interactive tasks. The advantage of using an asynchronous format was that learners and/or the instructor did not need to be online at the same time allowing the participant to work at his or her own pace. The asynchronous nature of this web based learning environment allowed for barriers of time, location and expense to be overcome. Sanoff (2005) 1.2 Moodle Description University College Londons (UCL) Moodle was the platform used to develop the educational resource. There are many applications offering free alternatives to the commercial software WebCTTM and BlackboardTM, however the UCL Moodle was chosen as a matter of convenience because it was accessible, independent of specific operating systems, fit for purpose and easy to use without much technical computer knowledge thereby potentially removing barriers to any future course design and development.. Moodle (Modular Object Oriented Dynamic Learning Environment) is software freely available to use and was developed by Dougiamas. Moodle. org (1999). The Moodle software was designed on pedagogical principles that encourage learner interaction in a virtual learning environment. Moodle is a course management system used to support Web-based courses and has a number of innovative tools that could be used to create courses that promoted collaborative learning. Moodle is able to run without modification on Unix, Linux, FreeBSD, Windows, Mac OS and Netware. (Moodle. Org). After an initial learning curve, the program was easy to use with simple but comprehensive online instructions. Moodle It did not require pre-existing computer programming knowledge, and in fact the author of this dissertation considered herself a novice computer user. Moodle is written in hypertext pre-processor (PHP) which is HTML embedded scripting language used to create dynamic Web pages.  PHP allows for connecting to remote servers, checking email, URL encoding and setting cookies. It offers good connectivity to many databases including MySQL, and PostgreSQL,  which Moodle uses as a single database. MySQL is a  relational database management system  that runs as a server providing multi-user access to a number of databases. (www.php.net). Moodle had the support for easily displaying multimedia aspects of the educational resource and the interface could be used in over 70 native language translations. The Web based educational resource was easily built up using multimedia activity modules and design elements, which included with easy navigation; Authentication and enrollment, Syndication with a chat forum made available to others as newsfeeds, Current evidence based didactic teaching, Interactive quizzes allowing import/export in a number of methods Hyperlinked resources to provide for branched learning, The use of a Wikipedia, A glossary of commonly used terms, Instructional video presentations. All the attributes of the Moodle made for an international transportable tool ideal for knowledge presentation, learner interaction, comments and reflection, dynamic and interactive assessments, flexibility, extendibility, and most importantly, support for autonomous learning and continued educational development around the world. The only noteworthy disadvantage of using the UCL Moodle was an imposed instructional design. 1.3 Resource Description The educational resource was named; VIRTUAL [emailprotected]: Onà ¢Ã¢â€š ¬Ã¢â‚¬Å"line Laryngeal Airway Device training. à ¢Ã¢â€š ¬Ã¢â‚¬Å" Virtual suggesting both the virtual reality of an Internet based generated environment and the adjective, meaning practically or almost Collins English Dictionary ( 2008). The Web based educational resource was developed for distance learning and contained all the elements of a totally Internet delivered educational resource. The content of the course was drawn from the authors personal experience using the Laryngeal device; peer reviewed journal articles, manufactures product information and videos downloaded from the Internet. Permission for the use of copywriter-restricted material was sought and obtained where appropriate. 1.3.1 Screen Design The screen design refers to how the information was arranged and presented on the display screen. The guidelines used followed those (amongst others) suggested by DoD HCI Style Guide (1992). The screen was kept simple, orderly, clutter free and consistent with a limited, non-dominating colour palate of four colours à ¢Ã¢â€š ¬Ã¢â‚¬Å" blue, black, white and blue à ¢Ã¢â€š ¬Ã¢â‚¬Å" green, a combination that has been shown to cause little fatigue and distraction. Kelley (1988). The content of each lesson was presented on a plain white background with black text in a non-jarring informal style font that made the lessons easy to read. Clark (1997). One template was used and the navigation bar, top bar and individual lesson heading bars kept the same with only the content of each lesson changing. All the content was displayed statically on one screen with individual lessons accessed by scrolling vertically down to the individual lesson. Unnecessary menus and long selection lists were avoid ed. This allowed for an overall view of the content, minimized pointer and eye movements and caused less distraction with easy navigation. (Gruneberg 1978). A discussion forum, interactive quizzes and an end of resource examination were included to allow engagement and self-assessment. The quizzes and examination included a range of question types à ¢Ã¢â€š ¬Ã¢â‚¬Å" multiple choice questions, true/ false, photo matching and random short answer matching type questions. These varieties of questions were shown to improve the learning experiences of adult learners. (Mackway-Jones, 1998). Information was provided in chunks and the writing style kept informal, with plain, simple language and in conversational tone with some elements of humour. There were fewer than 60 à ¢Ã¢â€š ¬Ã¢â‚¬Å" character positions on a standard 80 à ¢Ã¢â€š ¬Ã¢â‚¬Å" character line, spacing between characters were 25 à ¢Ã¢â€š ¬Ã¢â‚¬Å" 50% of character height and spacing between lines were equal to the character h eight, this to increase reading efficiency. 1.3.2 Course Content The course material was presented as text, graphics, power point presentations, hyperlinks and video demonstrations of the procedural skill presented in animation and on an actual patient. Knowledge was provided in five short lessons that followed the natural sequence of usage and intubation with a Laryngeal device. Aims of the resource and objectives of each lesson were stated at the beginning of the course. Here too a glossary of commonly used terms and a baseline knowledge assessment quiz were included. Each lesson was kept succinct with hyperlinks to websites and folders for those seeking extra information. This was to limit download times. Each lesson was concluded with an interactive quiz used to reinforce and test the knowledge learnt. Instant responses were provided to the quizzes after submission with suggestions to either revisit the lesson or to continue depending on the results obtained in the quizzes. The resource was concluded with an end of course examination and the c ourse was predicted to take 1 to 2 hours to complete. The resource content was accessed with a secured password with all content downloadable by way of an Internet connection. All the participants were supplied with a secure company email address and all the ISTCs had Internet access. Permission was requested for the use of company time and resources e.g. airway device training mannequin and time during the working day for those who chose to access the resource at work. Participants were supplied with instructions on how to use Microsoft Word and how to log on to and navigate the Moodle site. The course material was available online for two weeks with access monitored. 1.4 Pilot Study The aim of the pilot study was to assess the ease of navigation, gauge the time it took to complete the course, the integrity of the hyperlinks and the validity and reliability of the content and examination questions. Font preferences, layout and download speeds were also assessed. A prototype of the resource was tested on a selected sample of five participants of similar profile to the participants used in the study. The participants of the pilot study were excluded from participating in the actual study. An external panel of three Consultant Anaesthetists and two trainee Registrar Anaesthetists where used to provided expert advice. The Consultant Anaesthetists were selected based on their special interest in emergency medicine or difficult intubation scenario teaching. A few typographic errors were corrected, aims and objectives were clarified, difficult navigational issues were corrected and some content deemed repetitive and lengthy by the pilot participants were excluded before rolling out the resource. These changes however, were minor and further usability studies were deemed unnecessary. 1.5 Content Validation The content presented was current, evidence based and peer reviewed for content validity by the panel of experts (made up of three Consultant Anaesthetists and two trainee Registrar Anaesthetists), who deemed the content to be relevant and appropriate. The panel of experts and the pilot participants also judged good face validity. 2. Evaluation of the Resource The study evaluates effectiveness and acceptance of a Web based educational resource used to train and maintain a learnt procedural skill in the context of continuous medical Education (CME). The evaluation of the resource was undertaken in two parts. First the effectiveness of the resource was evaluated and the endpoints measured were changes in knowledge, confidence and technical ability. This evaluation made use of a summative framework redefined by Saettler (1990); which takes place after interaction with the resource. A before and after interventional ipsative assessment was undertaken where participants performance was compared to their own over a period of time. The second part of the evaluation was undertaken to assess the acceptability of the Web based educational resource as a medium for procedural skills training and this was done by way of an evaluation questionnaire completed by the participants after course completion. 2.1 Participants and Setting The participants and settings were specifically targeted, as they would ultimately be interested stakeholders and end users of this type of resource. The research was conducted at five Care UK TM Independent Centers (ISTCs) on practitioners employed at these facilities. The ISTCs are part of the governments initiative to reduce long NHS waiting times for elective surgery by adding increased capacity and alternative treatment venues for patients. There are approximately 25 ISTCs in the United Kingdom with Care UK TM represents 20% of this market. The ISTCs were chosen as a setting because: They are not part of the UK NHS medical training scheme and therefore have no formal programs of medical training or teaching that similar grades of staff in the NHS would receive. Contractual obligations of the ISTC contract decreed that the ISTCs could not employ medical practitioners from the NHS; therefore, most of the medical staff employed at the ISTCs have trained abroad and are waiting either to enter a formal career path within the NHS or wanting UK work experience. This situation has resulted in a mixture of nationalities, non-uniform medical training and medical staff with differing levels of post qualification experience and more importantly, a high staff turnover. (ref) These resulting factors were conducive to a system of competency-based appraisals and continuous medical education, which could be addressed with Web, based educational resources. Participants in the study were all frontline practitioners employed at Care UK TM ISTCs, which employs 48 practitioners of this grade. This represents 50% of all frontline practitioners employed in ISTCs throughout the UK. This intended sample size of 48 adequately represented the wider population in this type of analysis. frontline practitioners are the first practitioners on call to the resuscitation of a collapsed patient where they would be called upon to secure the patients airway and ensure oxygenation until the Anaesthetists or the resuscitation team arrives. It is expected that frontline practitioners are trained and certified with acute cardiac and life support skills and confident in dealing with clinical emergencies. In reality, analysis has shown that frontline practitioners in these ISTCs, though some trained and certified, rarely use these skills due to the infrequent nature of resuscitation clinical emergencies, making these scenarios potentially high-risk events when they do occur. Frontline practitioners are made up of Resident Medical Officers (RMOs), Anaesthetic Assistants (ODAs) and Recovery Room Practitioners (RNs). RMOs are doctors who have completed their medical training and have at least two years post graduate work experience as qualified doctors. They are employed to provide 24 hours on site medical management of patients at the ISTCs and like general practitioners (RACGP 2006) and doctors outside NHS academic hospitals, are usually first on call for emergencies and the sole source of medical advice on the premises on which they work. Anaesthetic assistants and recovery room practitioners are nursing practitioners