Guideline development process

Project background

In 2010, Cancer Council Australia embarked on the revision of the treatment and follow-up section of the 2004 Clinical Practice Guidelines for the Prevention, Diagnosis and Management of Lung Cancer as wiki-based, electronic guidelines with funding received from Cancer Australia. The revision covering Management and Follow-up was finalised in 2012 developed by the multi-disciplinary Lung Cancer Treatment Guidelines Working Party.

In 2012, the Management Committee of the Lung Cancer Guidelines Revision project was contacted to propose suitable lead authors and working party members for the prevention and diagnosis guidelines (see: Lung Cancer Treatment Guidelines Working Party).

In November 2012, Cancer Council Australia convened the first working party meeting to determine the included clinical questions to be part of the prevention and diagnosis section of the revised lung cancer guidelines.

Funding

The revised Clinical practice guidelines for the prevention and diagnosis of lung cancer are developed by Cancer Council Australia. No external funding has been received. Cancer Council Australia contributed in kind resources consisting of project staff, facilities, systems and travel budget to fund the prevention and diagnosis section revision of the 2004 lung cancer guidelines.

Back to top

Project governance, guidelines scope and guidelines development group

The small Management Committee appointed in 2009 is responsible to oversee the guidelines revision project. The Management Committee is responsible for the overall management and strategic leadership of the guidelines review process.

The Management Committee proposed lead authors for each included clinical question. The nominated individuals were invited to join (see multidisciplinary working party). In addition, the Management Committee identified and nominated a consumer representative to join the multidisciplinary working party.

In consultation with the question lead author, sub-committees consisting of members with relevant expertise and experience were established for each question (see multidisciplinary working party).

Declarations of interest were collected from all nominated members and evaluated (see COI register). All members were advised to update their declarations of interest over the course of the project and received reminders to review their declarations prior to every formal working party meeting.

Back to top

Guidelines development approach

The Management Committee agreed to use Cancer Council Australia’s Cancer Guidelines Wiki Platform and approach to develop the guidelines. The Wiki Platform is web-based and supports all processes of guidelines development, such as the literature search, critical appraisal, data extraction, evidence assessment and summary processes, as well as content and recommendation development, online consultation, review and web publication. It is in line with the NHMRC guidelines requirements, designated standards of quality, process and grading system for recommendations. An infrastructure is set in place to process literature updates and continuously update content as new evidence emerges and is reviewed.

The Development of Clinical Practice Guidelines using Cancer Council Australia’s Cancer Guidelines Wiki Handbook illustrates the steps in the development of Cancer Council Australia’s web-based clinical practice guidelines. It provides information to assist working party members and staff members to develop concise clinical questions in PICO format, construct sound search strategies, systematically search the literature, critically appraise, summarise the evidence and formulate guidelines recommendations.

Rather than waiting until systematic reviews and content for all included clinical questions have been finalised, the working party agreed to publish finalised question content and the associated recommendations in stages. The group decided that it is important to publish content and results as soon as it is finalised by the working party to ensure that the medical community receives up-to-date information without any publication delay. Prior to publication, feedback would be sought from guidelines stakeholders about the clinical questions content (See also Public Consultation).

The first set of completed draft contents regarding screening and early detection is now being released for public consultation:

• In people at risk of lung cancer, does population based screening with chest radiography reduce mortality?

• In people at risk of lung cancer, does population based CT screening reduce mortality?

• Which population group would most benefit from CT screening for lung cancer?

Subsequent clinical questions and associated recommendations will be published in 2017 and 2018.

The detailed steps in preparing the question content, conducting the literature searches, appraising the literature and formulating and grading recommendations, are outlined below.

Back to top

Steps in preparing clinical practice guidelines

For every clinical question the following steps were completed:

1. Develop a structured clinical question in PICO format

2. Search for existing relevant guidelines and systematic reviews answering the clinical question

3. Perform systematic review process

Developing the systematic review protocol and systematic literature search strategy for each PICO question

Conducting the systematic literature search according to protocol

Screening of literature results against pre-defined inclusion and exclusion criteria

Critical appraisal and data extraction of each included article

Body evidence table of all included literature

4. Summarise the relevant data

5. Assess the body of evidence and formulate recommendations

6. Write the content narrative

Back to top

Step 1. Develop a structured clinical question

All included questions were reviewed on the basis of their purpose, scope and clinical importance to the target audience and were structured according to the PICO (populations, interventions, comparisons, outcomes) framework. The lead authors provided the systematic review team with feedback to refine the PICO questions and inclusion and exclusion criteria for the systematic review.

Back to top

Step 2. Search for existing relevant guidelines and systematic reviews

For each PICO question, the National Guideline Clearinghouse, the Guidelines Resource Centre and the scoping search for the PICO question were scanned for relevant clinical practice guidelines to inform the question.

Back to top

Step 3. Perform systematic review process

Developing a systematic search strategy

For each PICO question, systematic literature search strategies were developed by the technical team. Searches were limited or widened as necessary according to the PICO structure using keywords or MESH and subject terms. Systematic search strategies were derived from these terms for each included electronic databases. The included standard databases searched were Pubmed, Embase, Trip database, Cochrane Database of Systematic Reviews and Database of Abstracts of Reviews of Effects and Health Technology Assessment for all questions. The psychosocial questions also included CINAHL and PsycINFO databases to retrieve relevant literature.

Back to top

Conducting the systematic literature search according to protocol

Clinical practice guidelines should be based on systematic identification and synthesis of the best available scientific evidence. For each clinical question that required a systematic literature review, literature searches were conducted systematically from 1990 onwards. The following electronic databases were part of the systematic literature search strategy:

• PubMed – bibliographic references and abstracts to articles in a range of languages on topics such as clinical medical information and biomedicine, and including the allied health fields, biological and physical sciences.

• EMBASE – major pharmacological and biomedical database indexing drug information from 4550 journals published in 70 countries.

• Trip Database – A medical database with focus on Evidence based medicine and clinical practice guidelines with content available from Cochrane and Bandolier.

• Database of Abstracts of Reviews of Effects and Health Technology Assessment – Contains details of systematic reviews that evaluate the effects of healthcare interventions and the delivery and organisation of health services.

• The Cochrane Database of Systematic Reviews.

• Cinahl – Bibliographic references and abstracts to journal articles, book chapters, pamphlets, audiovisual materials, software, dissertations, critical paths, and research instruments on topics including nursing and allied health, biomedicine, consumer health, health sciences librarianship, behavioral sciences, management, and education

• Psychinfo – Bibliographic references and abstracts to journal articles, book chapters, dissertations and technical reports on psychology; social, clinical, cognitive and neuropsychology; psychiatry, sociology, anthropology and education, with source material from a wide range of languages.

Additional relevant papers from reference lists and, where appropriate, clinical trial registries, were also identified for retrieval as part of the snowballing process.

The full detailed systematic literature search strategy for every clinical question is fully documented in the appendix of the clinical question.

Back to top

Screening of literature results against pre-defined inclusion and exclusion criteria

Part of the systematic review process is to screen all retrieved literature results against the pre-defined inclusion and exclusion criteria in two stages.

a) First screen – During the first screening round, the titles and abstracts of all retrieved literature were screened by the Project Officer. All irrelevant, incorrect and duplicates were removed.

b) Second screen – A second screen was undertaken based on the full article. The Project Officer and the allocated Lead Author assessed each article for inclusion against the pre-defined inclusion and exclusion criteria for each question. In the case of a disagreement between the reviewers, a third independent reviewer assessed the article against the inclusion and exclusion criteria. Articles that met the inclusion criteria were forwarded for quality assessment and data extraction to the lead author.

Back to top

Critical appraisal and data extraction of each included article

The risk of bias of each of the included studies was assessed using a study design specific assessment tool and where necessary pre-specified criteria. For all quality assessment tools, see Guideline Development Handbook.

For all included articles, the relevant data was extracted and summarised in study characteristics and evidence tables. Each risk of bias assessment and data extraction was checked by the Project Officer. These tables are available in the appendix of each question.

Back to top

Step 4. Summarise the relevant data

The study results, level of the evidence, risk of bias due to study design and the relevance of the evidence for each included study were summarised in a body of evidence table.

The NHMRC levels of evidence are outlined below:

Table 1. Designations of levels of evidence according to type of research question (NHMRC, 2009)

Level	Intervention	Diagnosis	Prognosis	Aetiology	Screening
I	A systematic review of level II studies	A systematic review of level II studies	A systematic review of level II studies	A systematic review of level II studies	A systematic review of level II studies
II	A randomised controlled trial	A study of test accuracy with: an independent, blinded comparison with a valid reference standard, among consecutive patients with a defined clinical presentation	A prospective cohort study	A prospective cohort study	A randomised controlled trial
III-1	A pseudo-randomised controlled trial (i.e. alternate allocation or some other method)	A study of test accuracy with: an independent, blinded comparison with a valid reference standard, among non-consecutive patients with a defined clinical presentation	All or none	All or none	A pseudo-randomised controlled trial (i.e. alternate allocation or some other method)
III-2	A comparative study with concurrent controls: Non-randomised, experimental trial Cohort study Case-control study Interrupted time series with a control group	A comparison with reference standard that does not meet the criteria required for Level II and III-1 evidence	Analysis of prognostic factors amongst untreated control patients in a randomised controlled trial	A retrospective cohort study	A comparative study with concurrent controls: Non-randomised, experimental trial Cohort study Case-control study
III-3	A comparative study without concurrent controls: Historical control study Two or more single arm study Interrupted time series without a parallel control group	Diagnostic case-control study	A retrospective cohort study	A case-control study	A comparative study without concurrent controls: Historical control study Two or more single arm study
IV	Case series with either post-test or pre-test/post-test outcomes	Study of diagnostic yield (no reference standard)	Case series, or cohort study of patients at different stages of disease	A cross-sectional study	Case series

Source: National Health and Medical Research Council. NHMRC additional levels of evidence and grades for recommendations for developers of guidelines. Canberra: NHMRC; 2009. (https://www.nhmrc.gov.au/_files_nhmrc/file/guidelines/developers/nhmrc_levels_grades_evidence_120423.pdf)

Back to top

Step 5. Assess the body of evidence and formulate recommendations

The body of evidence table for each clinical question was forwarded to the lead author for assessment. The lead author in collaboration with the Project Officer assessed the body of evidence and completed the evidence assessment matrix in regard to the volume of the evidence, its consistency, clinical impact, generalisability and applicability and developed evidence statements for each recommendation.

The process is described in NHMRC additional levels of evidence and grades for recommendations for developers of guidelines (2009).

Following grading of the body of evidence and development of evidence statements, authors were asked to formulate evidence-based recommendations based on the results of the systematic review summarised in the body of evidence table. The method of grading recommendations is shown in Table 2.

Table 2. Grading of recommendations

Component of Recommendation	Recommendation Grade
	A Excellent	B Good	C Satisfactory	D Poor
Volume of evidence 1**	one or more level I studies with a low risk of bias or several level II studies with a low risk of bias	one or two level II studies with a low risk of bias or a systematic review/several level III studies with a low risk of bias	one or two level III studies with a low risk of bias, or level I or II studies with a moderate risk of bias	level IV studies, or level I to III studies/systematic reviews with a high risk of bias
Consistency 2**	all studies consistent	most studies consistent and inconsistency may be explained	some inconsistency reflecting genuine uncertainty around clinical question	evidence is inconsistent
Clinical impact	very large	substantial	moderate	slight or restricted
Generalisability	population/s studied in body of evidence are the same as the target population for the guideline	population/s studied in the body of evidence are similar to the target population for the guideline	population/s studied in body of evidence differ to target population for guideline but it is clinically sensible to apply this evidence to target population3	population/s studied in body of evidence different to target population and hard to judge whether it is sensible to generalise to target population
Applicability	directly applicable to Australian healthcare context	applicable to Australian healthcare context with few caveats	probably applicable to Australian healthcare context with some caveats	not applicable to Australian healthcare context

¹ Level of evidence determined from level of evidence criteria

² If there is only one study, rank this component as ‘not applicable’

³ For example results in adults that are clinically sensible to apply children OR psychosocial outcomes for one cancer that may be applicable to patients with another cancer.

^**For a recommendation to be graded A or B, the volume and consistency of evidence must also be graded either A or B!

Source: National Health and Medical Research Council. NHMRC additional levels of evidence and grades for recommendations for developers of guidelines. Canberra: NHMRC; 2009. (https://www.nhmrc.gov.au/_files_nhmrc/file/guidelines/developers/nhmrc_levels_grades_evidence_120423.pdf)

The overall recommendations grade are shown in Table 3.

Table 3. Overall recommendation grades

Grade of recommendation	Description
A	Body of evidence can be trusted to guide practice
B	Body of evidence can be trusted to guide practice in most situations
C	Body of evidence provides some support for recommendation(s) but care should be taken in its application
D	Body of evidence is weak and recommendation must be applied with caution

Source: National Health and Medical Research Council. NHMRC levels of evidence and grades for recommendations for developers of guidelines. Canberra: NHMRC; 2009. (https://www.nhmrc.gov.au/_files_nhmrc/file/guidelines/developers/nhmrc_levels_grades_evidence_120423.pdf)

The NHMRC approved recommendation types and definitions are shown in Table 4.

Table 4. NHMRC approved recommendation types and definitions

Type of recommendation	Definition
Evidence-based recommendation	A recommendation formulated after a systematic review of the evidence, indicating supporting references
Consensus-based recommendation	A recommendation formulated in the absence of quality evidence, after a systematic review of the evidence was conducted and failed to identify admissible evidence on the clinical question
Practice point	A recommendation on a subject that is outside the scope of the search strategy for the systematic review, based on expert opinion and formulated by a consensus process

Source: National Health and Medical Research Council. Procedures and requirements for meeting the NHMRC standard for clinical practice guidelines. Melbourne: National Health and Medical Research Council, 2011

In addition to developing evidence-based recommendations as a result of the systematic review for a clinical question, expert authors could also draft consensus-based recommendations in the absence of evidence after having performed a systematic review or practice points, when a matter was outside the scope of the search strategy for the systematic review.

Back to top

Step 6. Write the content narrative

For each question, the assigned lead authors were asked to draft their guidelines chapter using the following format:

• Background to the clinical question, including its clinical importance and historical evidence, where relevant
• Review of the evidence, including the number, quality and findings of studies identified by the systematic review
• Evidence summary in tabular form including evidence statements, levels of evidence of included studies, and reference citations
• Evidence-based recommendation(s) and corresponding grade(s), consensus-based recommendations and practice points
• Discussion, including unresolved issues, relevant studies currently underway, and future research priorities
• References.

The content draft was then reviewed by the working party and feedback incorporated where required.

Back to top

Review of the draft chapters

Draft content was circulated to the working party. Members were asked to submit further suggestions on consensus-based recommendations and practice points.

A teleconference with all working party members was scheduled to review and finalise the draft content for public consultation. Prior to this teleconference, the latest iteration drafts were circulated. All panelists were asked to review the content, individual recommendations and practice points in detail, identify and note any controversies and points to be discussed at the meeting. During the meeting, each recommendation and practice point was tabled as an agenda point. Each was reviewed and approved by consensus, which was reached by voting. The Chairperson nominated a particular recommendation/practice point to be reviewed and the panelists had the opportunity to discuss any issues and suggest revisions to recommendations and practice points. Each recommendation and practice point was approved once the eligible panelists reached consensus.

Back to top

Public consultation

This guideline is being developed in a staged process.

• The first set of draft clinical questions (screening and early detection questions) was made available on the wiki for public consultation from 19 August to 19 September 2016.

• The second set of draft clinical questions was made available on the wiki for public consultation from 3-30 July 2017.

During each public consultation period, submissions are invited from the general public and professional societies and groups and other relevant stakeholders. Relevant professional societies and groups, consumer groups and other relevant stakeholders are contacted.

All feedback on the draft received during the consultation periods is compiled and sent to the relevant lead author (and subcommittee, when required) to review the draft content, assessing and considering the submitted comments. Any additional submitted papers are assessed by the methodologist team against the review protocol.

Wider Working Party review of the public consultation comments and suggested amendments is facilitated by email or teleconference. Subsequent changes to the draft are agreed by consensus, based on consideration of the evidence and, in the absence of evidence, expert opinion. The same consensus process that was followed during the face-to-face Working Party meeting prior to public consultation is followed.

All changes resulting from the public consultation submission reviews will be documented and made accessible by request once the guidelines are published.

Back to top

Dissemination and implementation

A multi-strategy approach will be followed for the dissemination and implementation of the guidelines, as this has shown to positively influence guidelines uptake.

Once all clinical questions that are part of the guidelines revision are completed, the guidelines will be distributed directly to relevant professional and other interested groups and through meetings, national and international conferences, and other professional development and continuing medical education (CME) events. Local expert leaders will be identified and approached to facilitate dissemination and act as champions for the guidelines.

A significant effort will be made to have the guidelines introduced to senior undergraduate medical students and to encourage the relevant learned colleges to support the guidelines and to foster their integration into hospital and community practice through resident and registrar education activities.

The guidelines will be made available as online guidelines via the Cancer Council Australia Cancer Guidelines Wiki. The online guidelines version increases availability as well as accessibility, and usage will be tracked and analysed with a web analytics solution. The Cancer Guidelines Wiki is a responsive website that is optimised for mobile and desktop access.

Interlinking and listing the guidelines on national and international guideline portal is also an important part of the digital dissemination strategy. Important Australian health websites, such as EviQ and healthdirect Australia will be approached to link to the online guidelines. The guidelines will also be listed on national and international guideline portals such as Australia’s Clinical Practice Guidelines Portal, Guidelines International Network guidelines library and National Guidelines Clearinghouse.

The Cancer Guidelines Wiki is based on semantic web technology, so the guidelines are available in a machine-readable format, which offers the possibility to easily integrate the guidelines content with systems and web applications used in the Australian healthcare context. Use of the guidelines as part of core curriculum in specialty exams will be encouraged.

It is recognised that a planned approach is necessary to overcome specific barriers to implementation in particular settings and to identify appropriate incentives to encourage uptake of guidelines recommendations. Implementation of the guidelines will require a combination of effective strategies and may include further CME initiatives and interactive learning, the development and promotion of computer-assisted decision aids and electronic decision-support systems, and the creation of audit and other clinical tools.

Back to top

Future updates

The handbook Development of Clinical Practice Guidelines Using Cancer Council Australia’s Cancer Guidelines Wiki. Handbook for section authors and the guideline working party. outlines Cancer Council Australia’s guidelines updating processes. The incoming literature updates will continue to be monitored for each systematic review question. The Working Party will notify the Project Officer if any clinical question requires revision because new high level evidence has been published. External stakeholders are encouraged to use the comment feature and notify us of any new evidence for a specific topic.

References

Back to top