Guideline development process

Guideline contents > Guideline development process

Guideline development process

Introduction

The Australian Cancer Network (ACN) initiated a proposal to develop the Clinical practice guidelines for the management of advanced prostate cancer. A decision to proceed was taken in September 2005. To better describe the scope of the guidelines, the title was changed to Clinical practice guidelines for the management of locally advanced and metastatic prostate cancer.

A Working Party composed of clinical specialists and consumers and a project team based in the Cancer Epidemiology Research Unit of Cancer Council New South Wales, carried out the work. The project team conducted literature searches, assisted in the critical evaluation of the literature and extracted the relevant data. Financial support was provided by Andrology Australia, the Prostate Cancer Foundation of Australia and ACN. NHMRC appointed a Guideline Assessment Reviewer (GAR) to both monitor and aid the development process.

The development program was designed to meet the scientific rigour required by the guideline development process, which is the subject of a series of handbooks on the main stages involved in the development of clinical practice guidelines. These handbooks have been previously condensed into a single volume—Development of clinical practice guidelines for the management of cutaneous melanoma and melanoma in special sites: a handbook for chapter leaders and expert working groups—which outlines the major steps and expectations involved in developing guidelines and provided a clear path for everyone involved in the project. This handbook provides the definitions and protocols for developing research questions and search strategies, conducting searches and critical appraisal, summarising and assessing the relevant literature and finally, formulating the recommendations. It includes checklists and templates created to satisfy designated standards of quality and process. These condensed handbooks have been a most useful aid in the demanding and, for some, new process of developing guidelines.

At its initial meetings the Guidelines Working Party prepared a table of topics and developed questions to address identified clinical needs. The questions were divided into different topics and subcommittees of the Guidelines Working Party were formed to address topics in their areas of expertise

Back to top

Steps in preparing clinical practice guidelines to NHMRC criteria

A clear strategy was developed for every topic and each expert group followed the appropriate steps in preparing the guidelines. While each subcommittee received significant assistance from the project team skilled in methodology, the subcommittees themselves oversaw the synthesis of the evidence and formulation of the recommendations for their topics.

The strategic steps followed are outlined below:

1. Structure the research questions

2. Develop a search strategy

3. Search the literature

4. Select, assess and summarise the literature

5. Critically appraise and summarise each selected article

6. Assess the body of evidence and formulate recommendations

Back to top

Structure the research questions

A wide range of questions was proposed for research. The questions focussed on interventions rather than diagnosis or prognosis. All proposed questions were reviewed on the basis of their purpose, scope and clinical importance to the target audience and were structured according to the PICO (populations, interventions, comparisons, outcomes) formulation.

The questions were ranked and then 52 were identified as requiring systematic reviews.

Back to top

Develop a search strategy

Each research question was submitted to a search strategy based on the PICO formulation. Most searches were directed to prostate cancer as a generic base. Searches were limited or widened as necessary, but all maintained the PICO structure. Keywords were selected during the PICO process. Further sources for keywords or MESH and subject terms were derived from evidence-based material, systematically reviewed articles and appropriately relevant literature. A single systematic search strategy was derived from these terms and applied to all included electronic databases.

Back to top

Search the literature

Clinical practice guidelines should be based on systematic identification and synthesis of the best available scientific evidence. All literature searches were conducted systematically using electronic databases concluding 1 April 2006. Examples include:

• Medline: bibliographic references and abstracts to articles in a range of languages on topics such as clinical medical information and biomedicine, and including the allied health fields, biological and physical sciences

• EMBASE: major pharmacological and biomedical database indexing drug information from 4550 journals published in 70 countries

• Cinahl: bibliographic references and abstracts to journal articles, book chapters, pamphlets, audiovisual materials, software, dissertations, critical paths, and research instruments on topics including nursing and allied health, biomedicine, consumer health, health sciences librarianship, behavioural sciences, management, and education

• Cochrane Library: regularly updated collection of evidence-based medicine databases, including The Cochrane Database of Systematic Reviews

• Clinical evidence: compendium of evidence on the effects of clinical interventions updated every six months published by the British Medical Journal Publishing Group

• Psychinfo: Bibliographic references and abstracts to journal articles, book chapters, dissertations and technical reports on psychology; social, clinical, cognitive and neuropsychology; psychiatry, sociology, anthropology and education, with source material from a wide range of languages.

For each search, the following details were provided in topic- or question-specific reports (available on request from the Australian Cancer Network):

• electronic databases searched

• terms used to search the databases

• search inclusion or exclusion criteria

• language

• study type

Studies published before 1 April 2006 could be included in the systematic reviews. Studies published after this date could not be included in the evidence base for the recommendations but could be referred to in the text and were described in the Appendices to the topic- or question-specific reports (available on request from the Australian Cancer Network). The project team also hand-searched the reference lists of the relevant articles to identify additional articles that had not been detected through searches of the electronic databases. Bi-annual meetings of the guidelines Working Party provided a forum for discussing and sharing overlapping evidence, the discovery of unpublished literature and information from other key organisations or individuals.

Back to top

Select, assess and summarise the literature

The literature identified by the electronic database searches was assessed for relevance to each question. The following steps were taken to select and sort the literature, with the details and results summarised in topic- or question-specific reports (available on request from the Australian Cancer Network):

1. Define the inclusion criteria

2. Review titles and abstracts of retrieved citations to identify potentially relevant articles

3. Obtain the full text of potentially relevant articles

4. Determine whether the study described in each collected article met the pre-defined inclusion criteria

5. Determine whether systematic reviews accounted for all preceding literature

6. Prepare folders to file searches, background papers and reviewed articles for each question addressed

Two independent assessors then assessed the quality of each of the included studies according to predefined criteria for the various study types. Any disagreements were adjudicated by a third reviewer.

The quality criteria were:

• randomised controlled trials (RCTs): blinding, allocation concealment, follow up and intentionto- treat analysis and mode of randomisation

• systematic reviews: search strategy used, the inclusion criteria and their application, study quality assessment, summary descriptive tables, pooling methods and examination of heterogeneity

• quasi-randomised and cohort studies: subject selection, group comparability, comparability of outcome measurement, blinding and completeness of follow up. Criteria for the critical appraisal process are available on the Australian Cancer Network website (<www.cancer.org.au/acn>). Summaries of the studies were tabulated in PICO format and the relevant data extracted and summarised in tables. The data extraction was checked by a second assessor. These tables of study haracteristics and evidence are included in the topic- or question-specific reports (available on equest from the Australian Cancer Network). The reports also contain lists of collected studies that dd not meet the inclusion criteria and the reason for their exclusion.

Back to top

Critical appraisal and summary

For each clinical question, the included studies and their results were summarised in a template (Template 1 in the Handbook). Each study was submitted to further critical appraisal. The level of the evidence, the quality of evidence as determined above, the size of effect and relevance of the evidence of each included study was documented.

Details of the templates, rating systems, and criteria for the critical appraisal process are available on the Australian Cancer Network website ([1]). Levels of evidence are outlined below.

Back to top

'Designations of levels of evidence for intervention research questions (NHMRC, 2005)'

Level	Intervention
I	A systematic review of level II studies
II	A randomised controlled trial
III-1	A pseudo-randomised controlled trial (ie alternate allocation or some other method)
III-2	A comparative study with concurrent controls: • non-randomised, experimental trial • cohort study • case-control study • interrupted time series with a control group
III-3	A comparative study without concurrent controls: • historical control study • two or more single-arm studies • interrupted time series without a parallel control group
IV	Case series with either post-test or pre-test/post-test outcomes

Source: Development of clinical practice guidelines for the management of cutaneous melanoma and melanoma in special sites: Handbook for chapter leaders and expert working groups, p18

Back to top

Assess the body of evidence and formulate recommendations

The body of literature was assessed by each expert sub-committee in regard to the volume of the evidence, its consistency, clinical impact, generalisability and applicability. These aspects were graded and documented in a second template (Template 2 in the Handbook).

Following grading of the body of evidence, expert sub-committees were asked to formulate a recommendation that related to the summarised body of evidence. This recommendation also had to be graded as follows:

Grade of Recommendation	Description
A	Body of evidence can be trusted to guide practice
B	Body of evidence can be trusted to guide practice in most situations
C	Body of evidence provides some support for recommendations but care should be taken in its application.
D	Body of evidence is weak and recommendation must be applied with caution

Back to top

Writing the chapter

All the expert sub-committees were asked to write their guidelines chapter using the following format:

• background

• review of the evidence

• evidence summary with levels of evidence and numbered references

• recommendation(s) and corresponding grade(s)

• references

Back to top

Review of the chapters

The body of evidence and recommendations for each chapter were reviewed by the Guidelines Working Party and final recommendations agreed to, based on the evidence.

Back to top

Public consultation

A complete draft of the guidelines was sent out for public consultation in Australia in September 2009. The consultation process included soliciting public review of the document through advertisement in a national newspaper, and alerting professional societies and groups and sponsors. All feedback on the draft received during the consultation period in Australia was reviewed by the Guidelines Working Party. Subsequent changes to the draft were agreed by consensus, based on consideration of the evidence. A final independent review of experts in their fields was conducted before the final draft was submitted for publication.

Back to top

Dissemination and implementation

The Australian Cancer Network will take the lead in disseminating the guidelines in Australia. This will include a campaign to raise awareness of the new guidelines that incorporates organised media coverage through multiple outlets and an official launch. The Guidelines will be distributed directly to relevant professional and other interested groups and through meetings, national conferences, and other CME events. A significant effort will be made to have the Guidelines introduced to senior undergraduate medical students and to encourage the relevant learned Colleges, which are usually binational (surgeons, radiation oncologists and pathologists), to support the Guidelines and to foster their integration into hospital and community practice through resident and registrar education activities.

The scope of implementation activities will depend on the availability of funding. Use of the Guidelines as part of core curriculum in specialty exams will be encouraged. It is recognised that a planned approach is necessary to overcome specific barriers to implementation in particular settings and to identify appropriate incentives to encourage uptake of guideline recommendations. Implementation of the Guidelines will require a combination of effective strategies and may include further CME initiatives and interactive learning, the development and promotion of computer-assisted decision aids and electronic decision-support systems, and the creation of audit and other clinical tools.

Back to top

Consumer feedback process for consumer version of clinical practice guidelines

A consumer document has been produced and is in its implementation phase. To ensure strong consumer input into the development of the consumer version of the Guidelines, we worked in partnership with the Prostate Cancer Foundation of Australia (PCFA). This organisation has a network of over 80 peer support groups in each state and territory. All peer support groups involve families as well as men with prostate cancer.

The PCFA nominated support group members for the consumer guide committee. Two members with advanced prostate cancer, Mr Trevor Hunt and Mr Don Baumber were nominated initially, and made enormously helpful contributions in the early stages. As we approached the final stages of development, two additional peer support members, Mr Bill McHugh and Max Shub were also appointed. Bill is immediate past Chair of the Support and Advocacy Committee of the Prostate Cancer Foundation of Australia.

When the final draft of the Guide was ready for feedback, Bill McHugh and Max Shub organised a process, so that all support group members in all states and territories could provide feedback on a section (three chapters) of the Guide. Because the Guide was very long (250 pages), it was thought that this was the best way to get feedback on all chapters in the Guide, but not overburden men and their families with too much to read. Feedback was then compiled by two medical members of the PCFA support group network and forwarded to the committee. The revised document was then reviewed again by these members and our consumer representatives.

When the consumer guide was launched, presentations were organised in Sydney, Melbourne and Brisbane at a PCFA event featuring a visiting international speaker, medical oncologist and prostate cancer survivor Dr Charles “Snuffy” Meyers. All of these events were well attended (200 or more attendees), providing discussion and feedback. The Guide was also launched at the Urological Society of Australia and New Zealand’s annual meeting in February 2010.

Widespread distribution to Urologists and other clinicians has been initiated by Andrology Australia and there has been enthusiastic uptake at the practice level. The document is available for download from the four major organisations, which are sources of prostate cancer information nationally (Andrology Australia http://www.andrologyaustralia.org, Prostate Cancer Foundation of Australia http://www.pcfa.org.au, Cancer Council Australia http://www.cancer.org.au/clinicalguidelines, Lions Australian Prostate Cancer website http://www.prostatehealth.org.au ).

The book has been welcomed by major consumer organisations.

Comments

Back to top