The ESMO-Magnitude of Clinical Benefit Scale (ESMO-MCBS) Working Group has looked into new ways to improve the validity of the ESMO-MCBS which is widely used to assess the magnitude of clinical benefit of anticancer medicines. Focusing on trials’ methodologies and quality, the analysis published in ESMO Open discusses ten issues that may affect study outcomes and identifies seven potential sources of bias that need to be addressed in the future revisions of the scale to avoid misrepresentations derived from flawed trial design, implementation and data analysis.
“Currently, the scale works under the assumption that the methodologies of the trials are valid and that trial implementation is high-quality. In recent years, we have found that this may not always be the case”, explains the leading author of the paper, Dr Bishal Gyawali, Queen’s University, Ontario, Canada.
Issues in study design, implementation, and data analysis that may influence study outcomes and compromise the ESMO-MCBS score
Achieving an unbiased scoring
The research group started its work to find out what specific shortcomings in the ESMO-MCBS could be fine-tuned as part of a commitment to constantly improve the validity of the score. Ten issues related to trial design, implementation and data analysis were reviewed and seven shortcomings were detected in the capacity of the ESMO-MCBS in its 1.1 version to identify and address biases.
According to Gyawali, new approaches could be incorporated to promote the validity of the scale in the future. "It will be ideal for the tool to evaluate the validity of the control arms in clinical trials, to assess the appropriateness of crossover and include criteria to determine the validity of non-inferiority design and margins", he explains. “An additional option will be to point and/or penalise studies where overall survival (OS) may have been overestimated due to sub-standard post-progression treatment, and to explicitly exclude scoring for post hoc subgroup findings.”
Among the other issues that may affect the validity of the trial results and thereby, the ESMO-MCBS scores are the reasons for individuals to leave trials, especially when they are not balanced between the study arms. "This is called informative censoring and is particularly important when the primary endpoint is not overall survival but a surrogate," continues Gyawali. "Finally, although ESMO-MCBS upgrades and downgrades the scores based on the effect on quality of life (QoL), it needs to incorporate a mechanism to identify or penalise trials where information regarding QoL is delayed or simply not published at all."
With regards to other issues such as early stopping of clinical trials, inflated response rates (RR) and durations in single-arm trials, no shortcomings in the ESMO-MCBS tool were identified. Stringencies are also currently in place for trials that use surrogate endpoint rather than overall survival.
A checklist for critically appraising trials
As an increasing number of new cancer therapies are approved, having an unbiased tool that can assess the magnitude of their clinical benefit becomes more and more relevant. However, the utility of the ESMO-MCBS, like all tools, lies in the hands of the user. In fact, the interpretation of the ESMO-MCBS scores requires critical appraisal of trials to understand caveats in trial design, implementation, and data analysis that may have biased results and conclusions.
"Lack of understanding of the various issues affecting clinical trial interpretation can lead to incorrect, although well intentioned, decisions in cancer care," highlights Gyawali. “Trainees and physicians in oncology globally are trained in various clinical skills and procedures but are unfortunately, not trained well in critical appraisal.” decisions in cancer care
Along with the identification of areas of improvement for the scale, the analysis also resulted in a ready-to-use resource for critical appraisal of oncology literature. “Our paper provides a checklist of ten items that although framed in the context of the scale can help the readers to correctly interpret and understand trials. This is also the reason why the paper is published as open-access,” continues Gyawali.
I hope that this checklist will also motivate all the stakeholders involved – i.e. funders, industry, researchers, trialists - to design and conduct trials that are in the best interest of the patients and advances the field of oncology
A dynamic tool
With the seven issues identified by the analysis, the ESMO-MCBS Working group aims to further fine-tune the scale. The goal is particularly relevant as the scale expands its use as a screening tool to identify cancer treatments that have potential therapeutic value warranting full evaluation for essential medicine listing.
In 2015 ESMO launched the first version of the ESMO-MCBS to facilitate improved decision-making regarding the value of anti-cancer therapies, promote the accessibility and reduce iniquity of access to high value cancer treatments. The updated version of the scale, known as ESMO-MCBS v1.1, was released in 2017. It incorporates ten revisions and allows for scoring of single-arm studies.
A revised version of the scale, ESMO-MCBS v2, and a scale to evaluate the clinical benefit of medicines for the treatment of haematological malignancies, ESMO-MCBS:H, will be soon launched.
For more information about the scale, please contact mcbs@esmo.org or visit us at ESMO-Magnitude of Clinical Benefit Scale.