Appraisal : a guide for medical practitioners
October 2003
Examples of best practice
There is no single ideal way to conduct an appraisal interview or collect the necessary inputs. Appraisal will be most effective when it is seen as an opportunity for doctors to appraise themselves, to review their learning and professional development and identify their own requirements.
Examples of available appraisal systems
Appointing appraisers
Two commonly adopted frameworks for appointing appraisers include the ‘Ballot and Bank’ framework and the ‘Line Manager or Hierarchical’ framework.
'Ballot and Bank' framework for appraisal
This model involves forming a bank of appraisers built upon a ratio of one appraiser for every six doctors. The bank is established through a ballot. All doctors are asked to nominate for the bank and an internal election is then conducted. The successful doctors become the first to be included in the bank and should expect to remain in the bank for between three to five years. The appraisee has the choice of three appraisers, and the same appraiser may be used from one year to the next at the discretion of both the appraiser and the appraisee. It is perceived as being more democratic, more about professional career development and about supporting doctors. However, it has also been criticised for being separate from job plan reviews and the business planning process [Go to reference 16].
'Line Manager or Hierarchical' framework for appraisal
This model follows the simple formula of the direct supervisor of a doctor at any level conducting the appraisal. In this system, appraisal is linked to job planning and business planning and risks being seen as a management tool, as performance review and as a threat to professional development and good clinical practice. The appraisee has little choice as to their appraiser. Further, by employing a line manager-led approach, appraisal risks becoming evaluative, instead of a reflective process aimed at developing individual effectiveness. The line manager system assumes a hierarchy that does not exist among senior doctors and would therefore not be appropriate. Furthermore, it is not flexible enough to be used by all specialties [Go to reference 16].
Carrying out the appraisal process
NHS appraisal toolkit
The NHS appraisal toolkit [Go to reference 27] is based on the principle that a single portal should be available to both appraising and appraisee GPs and consultants in the NHS. The toolkit (www.appraisals.nhs.uk) is a secure password protected site, which combines the benefits of using the electronic forms with accessibility via the internet. It is easy to navigate and gives access to advice, guidance, best practice and practical tools at any time and from any location. One of the key benefits of the toolkit is that it automatically generates a summary statement based on the information entered, including personal development plan (PDP), saving considerable time and effort during the preparation and appraisal itself.
According to doctors who have made use of the toolkit, the biggest attraction is the lack of paperwork and the ease with which sections can be completed. Information is more likely to be complete, legible and laid out logically, making the whole process more streamlined for both appraiser and appraisee. Doctors using the toolkit have completed the forms more thoroughly, with supporting documentation filed logically and in a more accessible format than those using the paper-based model. When completing the forms by hand, legibility can be an issue, changes are difficult to make and all entries need to be repeated every year for successive appraisals. Completing electronic versions of the forms means that information can be saved electronically, is legible and allows changes during preparation. Despite the many advantages, further refinements are still possible. The registration process could be simplified and some of the language is seen to be ‘educationalist jargon’. Nevertheless, the general conclusion is that the NHS appraisal toolkit serves as an excellent user-friendly model of appraisal for all doctors.
Collecting evidence
There are a number of ways in which to collect evidence for an appraisal. In addition, some appraisal schemes, like the 360 degree survey, are built around the collection of evidence.
The GMC recommends collecting evidence for appraisals under the following headings:
Good medical care
Maintaining good medical practice
Teaching and training
Relationships with patients
Working with colleagues [Go to reference 28].
360 degree surveys
The 360 degree survey is a technique used to collect evidence from those who work with the doctor, such as other medical colleagues, nurses, administrators and patients. The term 360 degree appraisal originated in the commercial sector and refers to ‘full circle’ feedback from bosses, peers and those more junior. It is also often referred to as ‘multi-source feedback’. The method evolved as the limitations of the more traditional ‘top-down’ approach became apparent—namely that it was perceived as unfair, biased, limited to one person’s perspective and often demotivating. The 360 degree appraisal has the potential to overcome these problems, and over the past decade it has been used extensively in industry and introduced in some general practices and hospital departments [Go to reference 29]. It has been suggested that using multiple sources, and applying a variety of methods to appraise doctors on multiple dimensions will improve the objectivity of the exercise. Feedback obtained from superiors, peers, juniors and patients is a good way to test interpersonal and communicative ability [Go to reference 30].
Methods of 360 degree appraisal in the health service are varied and include open ended, unstructured interviews, statements with a simple rating scale and structured questionnaires based on items from focus groups with GPs or consultants about what they consider to be indicators of good performance. It is argued that it is harder to discount the views of several colleagues or patients than the views of just one or two. However, it is time consuming to collect ratings from a range of people and it may be difficult to get a representative sample of patients. It is suggested that 360 degree surveys are most useful when used in combination with other sources of evidence.
In order for the 360 degree feedback method to be successful the information gathered must be fed back to the appraisee in a constructive and sensitive manner. It is best practice for the appraisee to go through the feedback with someone who can help them to interpret the results [Go to reference 31]. It is also important that there are resources for support following feedback, not least the need for mentoring and counselling. This support should be provided as soon as possible after giving the feedback, otherwise there is potential for negative consequences.
As with any type of appraisal, the outcome of the 360 degree process must be followed up. There is potential for the 360 degree process to be damaging for the individual and the organisation if it is done in isolation. The 360 degree appraisal must be facilitated and resourced. If it is not then the process will be viewed negatively because the expectation of change has not been fulfilled.
There are a number of principles that must be borne in mind if 360 degree appraisal is to be successfully introduced [Go to reference 29]:
- the tool must be well validated and easy to administer, analyse and interpret
- feedback must be anonymous
- it should be used only for developmental purposes, not performance management
- any decision affecting a doctor’s career should not be based on 360 degree feedback alone. It is part of a broader array of evidence about a doctor’s performance, from which appraiser and appraisee can identify overall patterns, themes and messages
- training must be given to appraisers and appraisees about how to make the most of the feedback.
Peer and 360 degree appraisal methods will clearly not work in dysfunctional departments and general practices. The culture of the organisation must be sufficiently open to deal with employees criticising and commenting on each other [Go to reference 33].
The DTI has produced detailed best practice guidelines on 360 degree appraisal which, while not being specifically targeted at medical professionals, contain a great deal of useful guidance [Go to reference 31].
Case studies of good practice
The following are examples of appraisal methods that have been adopted in different healthcare organisations.
Consultant peer appraisal
A system of consultant peer appraisal has been developed in the North Bristol Department of Anaesthesia that has explicit criteria, involves the use of portfolios and the collection of data from colleagues and a structured interview leading to the production of a personal development plan. The system seeks to combine support with challenge. A total of 117 criteria were identified by which a consultant anaesthetist could be appraised [Go to reference 34]. These cover five different areas of activity (eg clinical teaching) and five groups of attributes (eg attitudes, interpersonal skills). Only 12 criteria are specifically related to anaesthesia. Attitudes and team work are seen to be particularly important. Each consultant compiles a portfolio, which includes audit data from the hospital clinical information system as well as personal details, evidence of continuing education and other relevant material. Each doctor also nominates three or four colleagues who can be approached by the appraisers for comments about them (eg surgeons, theatre nurses, secretaries). All consultants in the group have previously attended a two-day appraisal training course, which includes interviewing and feedback skills. Each consultant selects two colleagues as appraisers, as evidence suggests that consultants choosing their own appraisers does not significantly alter the results of the appraisal [Go to reference 35]. The use of two appraisers allows more detailed analysis of the appraisee’s comments than might otherwise have occurred. In this instance, one appraiser from a different hospital site is assigned to each appraisee. The process has been proved to be motivating and preliminary evaluation shows that confidence among appraisers and appraisees has increased. The main concern with this approach is the time needed. This has to cover the construction and maintenance of a portfolio, the collection of colleagues’ views and an appraisal discussion involving two or three people lasting about 90 minutes. Effective accurate and easy-to-use hospital audit and information systems are essential. A climate of support rather than blame must exist if appraisal is to succeed [Go to reference 12].
Peer appraisal among GPs
Peer appraisal in most practices in the Northern Deanery had evolved from staff appraisal and its format is either one-to-one interviews or a group process involving doctors and sometimes the practice manager [Go to reference 36]. The content of peer appraisal varies a great deal and perceptions of its value are mixed. The three broad themes included in the appraisal: clinical areas, educational needs and the performance of the individual and the team, are clearly not mutually exclusive. While some practices seek to embrace all three within peer appraisal, most explicitly exclude clinical issues. Most practices that sought to address clinical issues found it difficult to define just what was being appraised in the very wide spectrum of clinical practice.
The process itself is undertaken differently by various practices. Some give all staff questionnaires to review the appraisee’s ‘team performance’ in terms of time-keeping, communication in the team, relationships with patients and so on. In other practices, the practice manager gathers data informally from team members on these issues and feeds back a composite assessment at the appraisal interview. The appraisee and sometimes the other practice partners, also fill in a pro-forma listing areas of good performance and areas in which competence could be increased. This combined information is discussed at the individual or group appraisal meeting and some form of personal learning or development plan are usually the outcome of the process. Advantages of peer appraisal include the beneficial effect on team functioning, cohesion and mutual support. Other benefits for individuals include the facilitation of personal development and stimulation of reading and reflection. Perceived disadvantages centre on time constraints and the difficulties inherent in a process that voluntarily submits doctors to self and peer criticism.
Away day appraisal
An effective approach to appraisal adopted by GPs general practitioners in Wallasey was an away-day [Go to reference 37]. Initial concerns among GPs surrounding appraisal included protected time, inadequate funding, and the lack of locums or cover within the practice. In response to these difficulties, the idea of a weekend away was suggested. Each appraiser was to carry out two or three appraisals in one day, reducing the time spent away from individual practices. Despite some initial reservation from GPs who thought that appraisal should be conducted in practice time, rather than at weekends, there was a good response. Appraisals averaged one and a half-hours in length and the PCT provided logistical support and organised funding. The feedback was very positive, with many choosing to attend for the whole day to take advantage of the networking and teambuilding opportunities associated with these meetings.
The consensus was that the away-day approach was a success as it offered a chance to accomplish appraisals in an efficient and positive manner, while ensuring that they did not encroach on practice time and result in a backlog of work. According to one doctor who had attended the away-day, the key to a successful appraisal process is providing enough protected time. He also suggested that the supportive environment of this away-day approach was one of several advantages. The fact that it was out of work time was not an issue, not least because the time was remunerated. More importantly, the whole process was not a rushed job in snatched time. The presence of colleagues who were also participating in their appraisal offered mutual support as well as a chance to socialise. However, a key aspect of the success of this approach was that this was a group of GPs who had worked well together for many years and had a very supportive management structure.
Implementation of a local system
Oxford Radcliffe Hospitals Trust, one of the largest trusts in the country, developed a four-stage process to the implementation of a local system of appraisal [Go to reference 24]. This system involved:
- finding out consultants’ views
- organising training
- producing a local version of the national policy and procedures
- agreeing a system for distributing, collating and archiving appraisal documents.