Before we describe the process of a systematic literature review, let us first clarify what we understand under the term “Systematic Literature Review” and why such a review is important.
Systematic Literature Reviews (SLR)
A Systematic Literature Review (SLR) is a structured and rigorous process to identify, appraise, analyze and synthesize all the scientific evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit and reproducible methods aimed at minimizing bias in the review process, in order to produce more reliable findings that can be used to inform decision making.
Systematic literature reviews are used in a variety of settings: they are a mainstay of evidence-based medicine and can be used to gather evidence on the effectiveness, prevalence and risk, etiology or economic evaluation of health interventions.
For medical devices, systematic review of the literature is needed as part of the clinical evaluation process, to assess the clinical data on the safety and performance of the device on the one hand and
establish the state of the art (SOTA) on the other hand in order to comply with the requirements of the EU Medical Devices Regulation 2017/745 (MDR).
For in vitro diagnostics, literature searches are a mandatory part of the performance evaluation process under the EU In Vitro Diagnostics Regulation 2017/746 (IVDR), with literature data contributing to demonstrating the scientific validity and clinical performance of the device.
For both types of devices, systematic literature searches contribute to establish the state of the art
(SOTA), which is understood to represent the current and generally accepted good practice in
technology and medicine (MDCG 2020-6) [1] .
Since clinical and performance evaluation need to be iterative and continuous processes , and as the SOTA in medicine and technology is rapidly evolving, systematic literature reviews for medical devices or IVDs likewise need to be updated regularly.
Therefore, setting up a reproducible process is imperative.
How to conduct a compliant Systematic Literature Review
Having defined the nature of a Systematic Literature Review, why it’s useful and required under the
current medical device regulations, we will now describe the steps to conduct an SLR.
Step 1. Define and validate your research question
The first step of the systematic review process is to define the objective and scope of your review. To do this, take into account the position of the device in its life cycle: whether it is a new, innovative device or a legacy device, as this will determine the amount of literature data that will be available. Make sure you are sufficiently familiar with the clinical domain of the device’s intended purpose to be able to select relevant keywords.
Literature reviews for medical devices will require at least two literature searches, one for clinical data on the device to assess its safety and performance and one for the SOTA on the clinical condition(s) to be treated or diagnosed.
For every search type, develop a clear, well-formulated research question, using a recognized structuring framework. While different structuring frameworks exist, the PICO framework is most commonly used for clinical and healthcare related questions. PICO stands for Population, Intervention, Comparison, Outcome and can be used to structure your search by creating a list of keywords or search terms for each of the PICO items.
Read about using the PICO framework or its alternatives in this blog post
A literature search protocol will be created to formalize the Literature Review methods. This protocol should include:
- Sources of data that will be used and a justification for their choice. To ensure sufficient coverage and minimize bias, at least two different databases need to be searched [2,3] .
- The search terms and exact search strategy for every search
- Clear inclusion and exclusion criteria and justification for their choice. Inclusion criteria are usually based on the PICO approach, study design, and publication date. Exclusion criteria comprise unrelated, duplicated, unavailable full texts, or abstract-only papers, in addition to specific exclusions related to the scope of the review. These exclusions must be stated in advance to prevent bias [4] . Methods to identify additional information via web searches and a strategy to handle information already held by the manufacturer.
- Strategies for identifying duplicate data.
Step 2. Database Search
Prior to proceeding, we would recommend a preliminary search to identify relevant articles, ensure the validity of the proposed search strategy and assure that we have enough articles for conducting its analysis. Usually, such initial search is done in PubMed.
Then create queries for all the searches you defined in your search protocol. A standard approach would be to create PubMed searches first, then translate search strings for other databases as needed. Search translation can either be done manually, or with the use of certain automation tools.
Step 3. Title and Abstract Screening
After performing your searches, you will first need to remove duplicates, i.e., identical articles that have been identified in different databases or searches, as well as different publications that cover the same data, such as a publication and a clinical trial record on the same trial.
Then, you will triage the results obtained from your searches, taking into account the eligibility criteria. If in doubt, you will include the article, rather than excluding it. According to the PRISMA flow diagram, reasons for exclusions are not strictly necessary in this step, but when performing a literature review for medical devices, the reason for exclusion must be documented for all search results, to ensure full traceability.
In case the number of search results is very large, a first pass screening on title only can be performed to quickly remove results that are obviously irrelevant, but in general screening of title and abstract is used to identify relevant articles.
Step 4. Full text Review
In the next step, obtain the full text of all included publications. Open access articles can be retrieved using the link out feature of literature search databases or via Unpaywall. In case the articles are not accessible for free, you should rent or buy it from the publisher or an article providing service. For full texts that cannot be located on the internet, the journals or authors should be contacted to obtain the articles. One has to exhaust all possibilities to obtain a selected article to reduce bias.
Similar to step 3, the full text articles of the included publications will be reviewed based on the in- and exclusion criteria. Reasons for exclusion of articles should also be documented. Full text review is needed for a final screening decision as the title and abstract often do not contain all information necessary on the inclusion and exclusion criteria of the review.
Upon your final selection of articles for review, the PRISMA flow diagram [5] can be constructed. The PRISMA flow diagram visually summarizes the screening process. It initially records the number of articles found and then makes the selection process transparent by reporting on decisions made at various stages of the systematic review.
Step 5. Critical Appraisal
During critical appraisal, reviewers formally and systematically appraise the quality of included studies. This process will help the reviewer to determine the value of the study and its results in the systematic literature review.
For evidence-based medicine and academic reviews, several appraisal frameworks have been defined, to estimate the level of evidence and risk of bias of a study, mostly based on the type of publication. For literature reviews on medical devices, Appendix F of IMDRF MDCE WG/N56 [6] provides a widely applicable generic set of appraisal criteria for suitability and data contribution.
Step 6. Data extraction
gather information to be used to categorize and summarize included studies, whereas quantitative data extraction is a process to systematically collect numerical data from relevant outcome parameters. These can then optionally be used in meta-analyses.
collection forms are invaluable in this process.
Step 7. Data analysis and Reporting
To make the results of your literature review easy to understand and use, present data in a tabular or graphical format as much as possible. When assessing device safety and performance, create summaries of relevant outcome parameters for the device under evaluation and any similar devices or alternative treatments.
If you have conducted one or more meta-analyses, you should present the results as an estimated effect across studies with a confidence interval. It is often simplest to show each meta- analysis summary with the actual results of includes studies in a forest plot.
Finally, you can write the Literature Review Report which describes the clinical background an state of the art and contains the results of your searches and analyses on safety, performance and clinical benefit, of the device under evaluation.
References and reading
- MDCG. MDCG 2020-6 Clinical evidence needed for medical devices previously CE marked under Directives 93/42/EEC or 90/385/EEC. 2020;1–22.
- Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non randomised studies of healthcare interventions, or both. BMJ. 2017;358:1–9. https://doi.org/10.1136/bmj.j4008
- European Commission. MEDDEV. 2.7.1 Rev.4: Clinical Evaluation: A Guide for Manufacturers And Notified Bodies Under Directives 93/42/EEC and 90/385/EEC. MEDDEV 271 Rev4. 2009;1–65.
- Tawfik GM, Dila KAS, Mohamed MYF, Tam DNH, Kien ND, Ahmed AM, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. Tropical Medicine and Health; 2019;47:1–9. https://doi.org/10.1186/s41182-019 -0165-6
- Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71.
- International Medical Device Regulators Forum. Clinical Evaluation. Imdrf Mdce Wg/N56Final2019. 2019.
