Guideline development process

From Cancer Guidelines Wiki

Introduction

These draft clinical practice guidelines are a revision and update of the 2005 Clinical Practice Guidelines for the Prevention, Early Detection and Management of ColorectalReferring to the large bowel, comprising the colon and rectum. Cancer. The guidelines were originally developed in 1999.

This current revision and update was commissioned and funded by the Department of Health Commonwealth of Australia.

The guideline project commenced in December 2014, and in June 2015 the National Health and Medical Research Council (NHMRC) agreed to consider approving the guideline, provided it was developed according to NHMRC procedures and requirements.

Back to top

Guidelines development group

Cancer Council Australia approached key stakeholders from the Working Party involved in the development of the 2005 colorectal cancer (CRC) guidelines. From this group, Cancer Council Australia appointed a designated Management Committee responsible for the overall management and strategic leadership of the guideline development process. This group acted as a steering committee to ensure that all deliverables agreed in the project plan were delivered to acceptable standards in accordance with NHMRC requirements, within agreed timeframes and within the approved budget.

A wider multidisciplinary Working Party of relevant experts was then convened to develop the revised guidelines and author specific sections. This was to ensure that representatives from all specialities and disciplines involved in the prevantion, diagnosis and management of CRC were represented. Two consumer representatives were invited to be part of the Working Party.

The guideline questions were allocated to specific guideline Working Party members to act as lead authors according to their areas of expertise. Each lead author team was able to co-opt additional experts as co-authors for their allocated questions. The Management Committee assessed the suggestion of any additional co-authors including their declaration of interest.

Back to top

Steps in preparing clinical practice guidelines to NHMRC criteria

A project team based at Cancer Council Australia conducted the systematic reviews, comprising of systematic literature searches, literature screening against pre-determined inclusion and exclusion criteria and critical evaluation and data extraction of the included literature. The project team was responsible for liaising with the Working Party members in regards to content development, content review and compiling the document. The clinical practice guideline was developed according to the procedures and requirements for meeting the 2011 NHMRC standard for clinical practice guidelines.[1] The development program was designed to meet the scientific rigour required by the standard for developing high quality, evidence-based clinical practice guidelines. A series of NHMRC resources and handbooks[2][3][4][5][6][7][8][9][10] guided the process and outlined the major steps and expectations involved in developing guidelines. These documents provided the definitions and protocols for developing research questions and search strategies, conducting systematic literature reviews, summarising and assessing the relevant literature and finally, formulating and grading the recommendations. They also included checklists and templates created to satisfy designated standards of quality and process.

For every question the below steps were followed:
1. Develop a structured clinical question (PICO question)
2. Search for existing relevant guidelines and systematic reviews

3. Process if relevant clinical practice guideline was identified or not
3a If no relevant clinical practice guideline was found


Check if an existing systematic review of high quality exists and can be used to inform the systematic review process


Developing the systematic review protocol and systematic literature search strategy for each PICO question


Conducting the systematic literature search according to protocol


ScreeningPerforming tests to identify disease in people before any symptoms appear. of literature results against pre-defined inclusion and exclusion criteria


Critical appraisal and data extraction of each included article

3b If a relevant clinical practice guideline was found and assessed as suitable for adaption

Conduct systematic literature review update for the question of the existing clinical practice guideline


ScreeningPerforming tests to identify disease in people before any symptoms appear. of literature update results against pre-defined inclusion and exclusion criteria


Critical appraisal and data extraction of each new included article


Update evidence table of evidence review of existing guideline with new literature update results



4. Summarise the relevant data
5. Assess the body of evidence and formulate recommendations
6. Write the content narrative
Back to top

Developing a structured clinical question

A wide range of questions were proposed for inclusion in the revised guidelines. In 2015, the Management Committee discussed the clinical questions that would be answered by systematic review. A shortlisting and voting process was undertaken to determine the final questions.

The questions focused on chemoprevention, screening, diagnosis, treatment and follow up. All proposed questions were reviewed on the basis of their purpose, scope and clinical importance to the target audience and were structured according to the PICO (populations, interventions, comparisons, outcomes) framework (see the clinical question list). The lead author and subcommittee members provided the systematic review team with feedback to refine the PICO questions.

Search for existing relevant guidelines and systematic reviews

For each PICO question, the National Guideline Clearinghouse, the Guidelines Resource Centre as well as the scoping search for the PICO question were scanned for relevant clinical practice guidelines that could potentially be suitable for adaption.

If an existing guideline was identified, the guideline was assessed for adaption according to the AGREEII assessment tool.

Relevant guidelines that did not meet the criteria for adaption were checked for systematic reviews that could be used as a source of relevant references to inform the systematic review process for the PICO question. Full systematic reviews were then performed as outlined in the following sections.

Back to top

Developing a systematic search strategy

For each PICO question, systematic literature search strategies were developed by the technical team. Most searches were directed to CRC as a generic base. Searches were limited or widened as necessary according to the PICO structure using keywords or MESH and subject terms. Systematic search strategies were derived from these terms for each included electronic databases. The included standard databases searched were PubMed, Embase, Cochrane Database of Systematic Reviews and Database of Abstracts of Reviews of Effects and Health Technology Assessment for all questions. The psychosocial questions also included CINAHL and PsycINFO databases to retrieve relevant literature.

Back to top

Conducting the systematic literature search according to protocol

Clinical practice guidelines should be based on systematic identification and synthesis of the best available scientific evidence.[2] For each clinical question, that required a systematic literature review, literature searches were conducted systematically with the literature cut-off date of 31 August 2016. The following electronic databases were part of the systematic literature search strategy:

  • PubMed (U.S. National Library of Medicine): bibliographic references and abstracts to articles in a range of languages on topics such as clinical medical information and biomedicine, and including the allied health fields, biological and physical sciences
  • EMBASE: major pharmacological and biomedical database indexing drug information from 4550 journals published in 70 countries
  • Database of Abstracts of Reviews of Effects and Health Technology Assessment: contains details of systematic reviews that evaluate the effects of healthcare interventions and the delivery and organisation of health services
  • The Cochrane Database of Systematic Reviews: contains systematic reviews of primary research in human health care and health policy, and are internationally recognised as the highest standard in evidence-based health care
  • CINAHL: bibliographic references and abstracts to journal articles, book chapters, pamphlets, audiovisual materials, software, dissertations, critical paths, and research instruments on topics including nursing and allied health, biomedicine, consumer health, health sciences librarianship, behavioural sciences, management, and education
  • Psychinfo: Bibliographic references and abstracts to journal articles, book chapters, dissertations and technical reports on psychology; social, clinical, cognitive and neuropsychology; psychiatry, sociology, anthropology and education, with source material from a wide range of languages.

A search filter to retrieve relevant literature considering Aboriginal and Torres Strait Islander peoples was added to each question.

Additional relevant papers from reference lists and, where appropriate, clinical trial registries, were also identified for retrieval as part of the snowballing process.

The full detailed systematic literature search strategy for every clinical question is fully documented in the technical report of the question (see Technical report).

ScreeningPerforming tests to identify disease in people before any symptoms appear. of literature results against pre-defined inclusion and exclusion criteria

Part of the systematic review process is to screen all retrieved literature results against the pre-defined inclusion and exclusion criteria in two stages.

a) First screen
During the first screening round, the titles and abstracts of all retrieved literature were screened by one or two reviewers. All irrelevant, incorrect and duplicates were removed.

b) Second screen
A second screen was undertaken based on the full article. A reviewer assessed each article for inclusion against the pre-defined inclusion and exclusion criteria for each question. In the case of a disagreement between the reviewers, a third independent reviewer assessed the article against the inclusion and exclusion criteria. Articles that met the inclusion criteria were forwarded for quality assessment and data extraction.

Critical appraisal and data extraction of each included article

Two assessors independently assessed the risk of bias of each of the included studies using a study design specific assessment tool and where necessary pre-specified criteria (see Technical report for all quality assessment tools). Any disagreements were adjudicated by a third reviewer.

For all included articles, the relevant data were extracted and summarised in study characteristics and evidence tables. Extracted data were checked by a second assessor. These tables are included in the technical report for each question (see Technical report).

Summary of the relevant data

For each outcome examined, the results, level of the evidence, the risk of bias due to study design, and the relevance of the evidence for each included study were documented in a body of evidence table.

Each question was addressed by a systematic review resulting in a systematic review report. All systematic review reports are published in the technical report of the guidelines. Levels of evidence are shown below.

Table A1. Designations of levels of evidence according to type of research question (NHMRC, 2009)

Level Intervention Diagnosis Prognosis Aetiology ScreeningPerforming tests to identify disease in people before any symptoms appear.
I A systematic review of level II studies A systematic review of level II studies A systematic review of level II studies A systematic review of level II studies A systematic review of level II studies
II A randomised controlled trial A study of test accuracy with: an independent, blinded comparison with a valid reference standard, among consecutive patients with a defined clinical presentation A prospective cohort study A prospective cohort study A randomised controlled trial
III-1 A pseudo-randomised controlled trial (i.e. alternate allocation or some other method) A study of test accuracy with: an independent, blinded comparison with a valid reference standard, among non-consecutive patients with a defined clinical presentation All or none All or none A pseudo-randomised controlled trial (i.e. alternate allocation or some other method)
III-2 A comparative study with concurrent controls:

Non-randomised, experimental trial

Cohort study

Case-control study

Interrupted time series with a control group

A comparison with reference standard that does not meet the criteria required for Level II and III-1 evidence Analysis of prognostic factors amongst untreated control patients in a randomised controlled trial A retrospective cohort study A comparative study with concurrent controls:

Non-randomised, experimental trial

Cohort study

Case-control study

III-3 A comparative study without concurrent controls:

Historical control study

Two or more single arm study

Interrupted time series without a parallel control group

Diagnostic case-control study A retrospective cohort study A case-control study A comparative study without concurrent controls:

Historical control study

Two or more single arm study

IV Case series with either post-test or pre-test/post-test outcomes Study of diagnostic yield (no reference standard) Case series, or cohort study of patients at different stages of disease A cross-sectional study Case series
Source: National Health and Medical Research Council. NHMRC additional levels of evidence and grades for recommendations for developers of guidelines. Canberra: NHMRC; 2009. (https://www.nhmrc.gov.au/_files_nhmrc/file/guidelines/developers/nhmrc_levels_grades_evidence_120423.pdf)

Back to top

Assess the body of evidence and formulate recommendations

The technical report for each question was forwarded to each lead author. The authors, in collaboration with their subcommittee members and systematic review team (who conducted the systematic reviews and provided the technical reports), assessed the body of evidence and completed the NHMRC Evidence Statement form to record the volume of the evidence, its consistency, clinical impact, generalisability and applicability and developed evidence statements (see Technical report). The process is described in NHMRC additional levels of evidence and grades for recommendations for developers of guidelines (2009).[10]

Following grading of the body of evidence and development of evidence statements, expert authors were asked to formulate evidence-based recommendations that related to the summarised body of evidence. The method of grading recommendations is shown in Table A2.

Table A2. Grading of recommendations

Component of Recommendation Recommendation Grade
A

Excellent

B

Good

C

Satisfactory

D

Poor

Volume of evidence 1** one or more level I studies with a low risk of bias or several level II studies with a low risk of bias one or two level II studies with a low risk of bias or a systematic review/several level III studies with a low risk of bias one or two level III studies with a low risk of bias, or level I or II studies with a moderate risk of bias level IV studies, or level I to III studies/systematic reviews with a high risk of bias
Consistency 2** all studies consistent most studies consistent and inconsistency may be explained some inconsistency reflecting genuine uncertainty around clinical question evidence is inconsistent
Clinical impact very large substantial moderate slight or restricted
Generalisability population/s studied in body of evidence are the same as the target population for the guideline population/s studied in the body of evidence are similar to the target population for the guideline population/s studied in body of evidence differ to target population for guideline but it is clinically sensible to apply this evidence to target population3 population/s studied in body of evidence different to target population and hard to judge whether it is sensible to generalise to target population
Applicability directly applicable to Australian healthcare context applicable to Australian healthcare context with few caveats probably applicable to Australian healthcare context with some caveats not applicable to Australian healthcare context
1 Level of evidence determined from level of evidence criteria
2 If there is only one study, rank this component as ‘not applicable’
3 For example results in adults that are clinically sensible to apply children OR psychosocial outcomes for one cancer that may be applicable to patients with another cancer.
**For a recommendation to be graded A or B, the volume and consistency of evidence must also be graded either A or B.

Source: National Health and Medical Research Council. NHMRC additional levels of evidence and grades for recommendations for developers of guidelines. Canberra: NHMRC; 2009. (https://www.nhmrc.gov.au/_files_nhmrc/file/guidelines/developers/nhmrc_levels_grades_evidence_120423.pdf)

The overall recommendations grade are shown in Table A3.

Table A3. Overall recommendation grades

Grade of recommendation
Description
A
Body of evidence can be trusted to guide practice
B
Body of evidence can be trusted to guide practice in most situations
C
Body of evidence provides some support for recommendation(s) but care should be taken in its application
D
Body of evidence is weak and recommendation must be applied with caution
Source: National Health and Medical Research Council. NHMRC levels of evidence and grades for recommendations for developers of guidelines. Canberra: NHMRC; 2009. (https://www.nhmrc.gov.au/_files_nhmrc/file/guidelines/developers/nhmrc_levels_grades_evidence_120423.pdf)

In addition to developing evidence-based recommendations as a result of the systematic review for a question, expert authors could also draft consensus-based recommendations in the absence of evidence after having performed a systematic review, or practice points, when a matter was outside the scope of the search strategy for the systematic review. The NHMRC approved recommendation types and definitions are shown in Table A4.

Table A4. NHMRC approved recommendation types and definitions

Type of recommendation
Definition
Evidence-based recommendationA recommendation formulated after a systematic review of the evidence, indicating supporting references.
A recommendation formulated after a systematic review of the evidence, indicating supporting references
Consensus-based recommendationA recommendation formulated in the absence of quality evidence, after a systematic review of the evidence was conducted and failed to identify admissible evidence on the clinical question.
A recommendation formulated in the absence of quality evidence, after a systematic review of the evidence was conducted and failed to identify admissible evidence on the clinical question
Practice pointA recommendation on a subject that is outside the scope of the search strategy for the systematic review, based on expert opinion and formulated by a consensus process.
A recommendation on a subject that is outside the scope of the search strategy for the systematic review, based on expert opinion and formulated by a consensus process
Source: National Health and Medical Research Council. Procedures and requirements for meeting the NHMRC standard for clinical practice guidelines. Melbourne: National Health and Medical Research Council, 2011

Back to top

Writing the content

For each clinical question, the assigned lead authors were asked to draft their guideline chapter using the following format:

  • general introduction to the clinical question
  • background to the clinical question, including its clinical importance and historical evidence, where relevant
  • review of the evidence, including the number, quality and findings of studies identified by the systematic review
  • evidence summary in tabular form including evidence statements, levels of evidence of included studies, and reference citations
  • evidence-based recommendation(s) and corresponding grade(s), consensus-based recommendations and practice points
  • implications for implementation of the recommendations, including possible effects on usual care, organisation of care, and any resource implications
  • discussion, including unresolved issues, relevant studies currently underway, and future research priorities
  • references.

For sections not based on systematic review, the lead author was asked to draw on high-level evidence, particularly international guidelines, consensus statements and key literature considered to be relevant to Australian practice, to develop information and practice points.

The content draft was then reviewed by subcommittee members who were available. The draft documents often underwent several iterations.

Back to top

Review of the draft chapters

The draft guideline sections were circulated to the Working Party members and posted on Cancer Council’s wiki platform. The group was asked to review the content and submit feedback. Members were asked to submit further suggestions on consensus-based recommendation and practice points.

A face-to-face meeting with all available Working Party members was held in December 2016 to review and finalise the draft guidelines for public consultation. Prior to this meeting, the latest version of the draft guideline was circulated as soon as they were available. All members were asked to review the content, individual recommendations and practice points in detail, and to identify and note any controversies and points to be discussed at the group meeting.

During the meeting, each chapter/section was tabled as an agenda point and recommendations and practice points were discussed in detail. All clinical guidance was reviewed and approved by consensus, which was reached by voting. In some cases, the authors agreed on specific actions for the content or discussed further sections or amendments to be added. These were actioned by the authors.

Each recommendation and practice point was approved once the eligible panellists (excluding representatives of the funding bodies and panellists who cannot vote due to conflict of interest) reached consensus. See the administrative report for information on conflict of interest declarations and action required.

Areas of major debate

There was major debate and robust discussion within the Working Party and/or subcommittee members on the following chapters:

  • Primary prevention (Chemopreventive candidate agents [PPR1 – aspirin systematic review]) There was robust discussion within the chapter subcommittee regarding the clinical background of the participants in the reported randomised controlled trials; the gender imbalance across these trials; and the potential harms and benefits of taking aspirin, both in the context of colorectal cancer prevention, prevention of other cancers, and the role of aspirin in preventing cardiovascular events. However the group was able to come to a decision about the guidance in this chapter, based on the interpretation of the systematic review evidence.
  • The symptomatic patient: Optimal maximum time from referral to diagnosis and treatment (SPT1-2b systematic review) The Working Party and subcommittee members had robust discussion regarding the maximum optimal time from first healthcare presentation to diagnostic colonoscopy and treatment. Although the group was in agreement about the interpretation of the systematic review evidence, there was concern about de-emphasising the need for prompt evaluation. The Working Party acknowledges that the guideline may be read with the expectation that it will assist in triage of colonoscopy patients. The authors resolved it was appropriate to maintain the evidence-based recommendations, acknowledging the grade and limitations of the available evidence, but also add the practice point about the ideal interval for symptomatic patients.
  • Risk and screening based on family history: Colorectal cancer risk according to family history (FHS2) There was robust discussion by the Working Party about the categories of risk outlined in this chapter. For Category 3, there was discussion regarding the decision to exclude people known to have, or with a high probability of having, a high-risk familial syndrome due to a genetic predisposition to colorectal cancer. Ultimately the Working Party was in agreement about the three-level risk categorisation and feel this is adequately outlined in the chapter.

In each instance, the guideline development working group was able to reach a decision about the content and recommendations.

Back to top

Public consultation

A complete draft of the guideline was sent out for public consultation from 10 March 2017 to 8 April 2017. Submissions were invited from the general public and professional societies and groups and other relevant stakeholders. The consultation was publicised by email to key stakeholders, including contacting professional societies and groups, consumer groups and other relevant parties.

All feedback on the draft received during the consultation period was compiled and sent to the relevant author and subcommittee to review their draft content, assessing and considering the submitted comments. Each additional submitted paper during public consultation was assessed by the methodologist team against the systematic review protocol to determine if it could be included.

Another face-to-face Working Party meeting was held in April 2017 to review all public consultation comments and the amended guideline content. Subsequent changes to the draft were agreed by consensus, based on consideration of the evidence. The same consensus process that was followed prior to public consultation would be followed again. All changes resulting from the public consultation submission reviews were documented and will be made accessible once the guideline is published.

A final independent review was conducted before the final draft was submitted to NHMRC Council. Further suggestions by the independent expert reviewers were considered and integrated in the final draft and then submitted to NHMRC Council for approval.

Back to top

Organisations formally endorsing the guidelines

The following medical colleges and professional bodies may be approached to endorse the guideline:

  • Australian College of Rural and Remote Medicine (ACRRM)
  • Medical Oncology Group of Australia Incorporated (MOGA)
  • Royal College of Pathologists of Australia (RCPA)
  • Royal Australasian College of Physicians (RACP) – Adult Medicine Division
  • Royal Australian College of Physicians – Australian Chapter of Palliative Medicine (AChPM, RACP)
  • Royal Australian College of Physicians – Australian Faculty of Public Health Medicine (AFPHM, RACP)
  • Royal Australian College of Surgeons (RACS)
  • Royal Australian College of General Practitioners (RACGP).

Back to top

Dissemination and implementation

Cancer Council Australia have created a plan regarding the dissemination of the guideline in Australia.

The guideline will be made available online via the Cancer Council Australia Cancer Guidelines wiki. The online guideline version increases availability as well as accessibility, and usage will be tracked and analysed with a web analytics solution. Interlinking and listing the guidelines on national and international guideline portal is an important part of the digital dissemination strategy. Important Australian health websites, such as EviQ and healthdirect Australia will be approached to link to the online guideline. The guideline will also to be listed on national and international guideline portals such as Australia’s Clinical Practice Guidelines Portal, Guidelines International Network guidelines library and National Guidelines Clearinghouse. The Cancer Guidelines wiki is a responsive website that is optimised for mobile and desktop access. When accessing the guidelines with a mobile and tablet device, an icon can be easily added to the home screen of mobile devices, offering easy mobile access.

In addition, the final guideline document will be launched via email alert to professional organisations, interested groups and clinical experts in the field, directing them via URL link to the online guideline and all associated resources.

The Cancer Guidelines wiki is based on semantic web technology, so the guidelines are available in a machine-readable format, which offers the possibility to easily integrate the guideline content with systems and web applications used in the Australian healthcare context.

Use of the guideline as part of core curriculum in specialty exams will be encouraged. It is recognised that a planned approach is necessary to overcome specific barriers to implementation in particular settings and to identify appropriate incentives to encourage uptake of guideline recommendations. Implementation of the guidelines will require a combination of effective strategies and may include further CME initiatives and interactive learning, the development and promotion of computer-assisted decision aids and electronic decision-support systems, and the creation of audit and other clinical tools.

Back to top

Future updates

The incoming literature updates will continue to be monitored for each systematic review question. If there is strong evidence emerging in a specific area of colorectal cancer management, the Management Committee will be reconvened to assess if this warrants a guideline update (full or partial). It is recommended that the guideline be updated after 5 years.

References

  1. National Health and Medical Research Council. Procedures and requirements for meeting the NHMRC standard for clinical practice guidelines. Melbourne; 2011.
  2. 2.02.1 National Health and Medical Research Council. A guide to the development, evaluation and implementation of clinical practice guidelines. Commonwealth of Australia: National Health and Medical Research Council; 1999 Jan 1 Abstract available at http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp30.pdf.
  3. National Health and Medical Research Council. How to review the evidence: Systematic identification and review of scientific literature. Canberra: National Health and Medical Research Council; 1999 Available from: http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp65.pdf.
  4. National Health and Medical Research Council. How to prepare and present evidence-based information for consumers of health services: A literature review. Commonwealth of Australia: National Health and Medical Research Council; 1999 Jan 1 Abstract available at http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp72.pdf.
  5. National Health and Medical Research Council. How to present evidence for consumers: Preparation of consumer publications. Canberra: Commonwealth of Australia; 1999.
  6. National Health and Medical Research Council. How to put evidence into practice: Implementation and dissemination strategies. Commonwealth of Australia: National Health and Medical Research Council; 2000 Jan 1 Abstract available at http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp71.pdf.
  7. National Health and Medical Research Council. How to use the evidence: assessment and application of scientific evidence. Commonwealth of Australia: National Health and Medical Research Council; 2000 Jan 1 Abstract available at http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp69.pdf.
  8. National Health and Medical Research Council. How to compare the costs and benefits: evaluation of the economic evidence. Commonwealth of Australia: National Health and Medical Research Council; 2001 Jan 1 Abstract available at http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp73.pdf.
  9. National Health and Medical Research Council. Using socioeconomic evidence in clinical practice guidelines. NHMRC 2002 Abstract available at http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp89.pdf.
  10. 10.010.1 National Health and Medical Research Council. NHMRC additional levels of evidence and grades for recommendations for developers of guidelines. Canberra; 2009 Available from: https://www.nhmrc.gov.au/_files_nhmrc/file/guidelines/developers/nhmrc_levels_grades_evidence_120423.pdf.
Back to top