Cancer clinical trials: identifying & testing approaches to impact evaluation

Hanna, Catherine Ruth (2021) Cancer clinical trials: identifying & testing approaches to impact evaluation. PhD thesis, University of Glasgow.

Abstract

Most cancer research is performed with the aim of generating new knowledge that leads to benefits such as improved treatments, higher cure rates, and better cancer prevention. Evaluating these downstream effects of research, often referred to as research impact, is of increasing importance to all cancer research stakeholders. There is currently no consensus surrounding the optimal way to approach this evaluation. The work in this thesis aimed to address this gap by first identifying which approaches to impact assessment have been applied previously for cancer research, and in particular for cancer clinical trials, and secondly to test a number of these approaches within the context of a case study of one cancer clinical trial. The Short Course Oncology Treatment (SCOT) trial was chosen for the purposes of the case study.

SCOT was a phase III randomised controlled trial (RCT) which tested the non-inferiority of shortening adjuvant treatment for patients with colorectal cancer (CRC) from the standard of 6 months to 3 months of doublet chemotherapy. The trial met its pre-specified non-inferiority end-point but showed unexpected differences in outcome based on the treatment regimen used and stage of disease. Specifically, for patients receiving CAPOX (capecitabine and oxaliplatin), non-inferiority for 3 months versus 6 months of treatment was met, but this was not the case for those treated with FOLFOX (5-fluorouracil and oxaliplatin). Similarly, non-inferiority was met for patients with small tumours with a small nodal burden (low-risk stage III), but not for those with more extensive disease and/or a higher nodal burden (high-risk stage III disease). SCOT was the largest contributor to a collaboration of six trials addressing the same research question, called the International Duration Evaluation of Adjuvant therapy (IDEA).

A systematic literature review was used to identify methods, frameworks, and categories of impact frequently used to perform research impact assessment (Chapter 3). This review was also used to identify previous impact assessments specific to cancer research. Fourteen empirical examples were identified, published between the years 1996 to 2015. These included assessment of research at the cancer project, programme and research centre level. One example specifically assessed the impact of a phase III cancer RCT. The methods for impact analysis included across these examples included surveys, interviews, bibliometric searching of journals and clinical guidelines, economic approaches and documentary analysis. The categories of impact most commonly used were policy, clinical practice, health and economic impact. The Payback framework and the Canadian Academy of Health Sciences (CAHS) framework were utilised to collect data and communicate the results of impact assessment in two of these examples.

A second approach was adopted to identify ways that have previously been used to assess the impact of cancer clinical trials in particular (Chapter 4). The research impact case studies submitted to the United Kingdom government’s research performance exercise for universities in 2014 were screened to find examples of assessments of the impact of cancer clinical trials. In total, 46 case studies describing 110 clinical trials were identified. Many of these trials were phase III trials that met their primary endpoint, but earlier phase trials and those with negative findings were also impactful. Policy impact was the most commonly described downstream effect. There was a gap within these case studies in the use of real world evidence to demonstrate the impact of cancer trials on clinical practices and health.

A number of the approaches to impact assessment identified in the literature review and in the analysis of the REF 2014 case studies were then tested to evaluate the impact of the SCOT Trial. The methods used for this assessment included surveys of clinician prescribing practices (Chapter 5), economic evaluation of the budget impact of trial results implementation (Chapter 6), and interrogation of real world data to explore the clinical practice and potential health benefits attributable to the SCOT trial at both a local (Chapter 7) and national (Chapter 8) level.

A clinician survey performed in April 2019 demonstrated a high level of awareness of SCOT trial results (Chapter 5), with 98% of those who were aware of the trial indicating they had changed their clinical practice based on the trial results. This impact on practice was driven mainly by shortening of treatment to 3 months for patients with low-risk stage III CRC (SCOT non-inferiority met), whereas most clinicians reported they still used 6 months of doublet chemotherapy for patients with high-risk stage III disease (SCOT non-inferiority not met). This finding aligned with the post-hoc subgroup analysis performed for both the SCOT trial and IDEA collaboration. When shortening treatment for this subgroup of patients, clinicians mainly used CAPOX, whereas there was a more even split between using CAPOX and FOLFOX when 6 months of treatment was still used. A follow up survey in August 2020 was performed using a subset of respondents to the first survey and showed an increase in the use of shorter (3 months) treatment for patients with stage III disease with one high-risk feature, compared to responses in April 2019.

The results of the first clinician survey were applied within a budget impact analysis (Chapter 6) to estimate the economic impact of implementing SCOT trial results in the six countries that recruited to the trial. It was estimated that implementation of SCOT trial findings could translate to over $150 million USD savings over five years for those 6 healthcare systems (Australia, Denmark, New Zealand, Spain, Sweden, UK). Adopting a societal perspective by including money lost because patients did not work when receiving longer treatment, as well as travel costs to hospital, increased this impact to $340 million USD. Adding the monetised quality adjusted life-year (QALY) gains from implementation to this calculation ($456 million USD) meant that the gains from implementation of SCOT were vastly in excess of the original investment to conduct the SCOT trial ($8.8 million USD).

The final analysis conducted as part of the SCOT case study involved examination of individual patient level chemotherapy prescribing data. Using local (one health board in Scotland) level data, five different approaches were tested to evaluate the impact of the SCOT trial. In this instance, the change in practice was obvious even using simple descriptive statistics. Out of the other methods tested, interrupted time series analysis (ITSA) was the additional method that added the most value; the strengths of the ITSA were the ability to visualise the trends in prescribing pre and post-SCOT, as well as the counterfactual situation. Focusing on patients prescribed doublet chemotherapy (as per the SCOT trial), there was a significant decrease (85% to 31%) in the proportion of patients receiving over 3 months of treatment after the SCOT trial results were published (ꭓ2 p<0.001) compared to before this time-point. There was no significant change in a comparator group of patients who received monotherapy (76% pre-SCOT versus 77% post-SCOT (ꭓ2 p=0.774)).

In order to evaluate this impact at a national level, it was first necessary to establish, for the first time, linkage of chemotherapy prescribing data at a pan-Scottish level. This process presented several challenges relating to data access, resource and infrastructure. Analysis of this data demonstrated a reduction in the proportion of patients receiving over 3 months of treatment across cancer networks in Scotland, although this change was less marked for patients treated in the Northern cancer network because 3 months of treatment was used proportionally more in the pre-SCOT period, compared to in the West or South-East of the country. The change in practice across the country was driven by changes for patients receiving CAPOX specifically, rather than FOLFOX or monotherapy, again fitting with the SCOT and IDEA subgroup analyses. Change was also greater for patients with low-risk rather than high-risk stage III disease, mirroring the clinician survey results. Following these in-depth analyses across Chapters 5-8, results from the survey, economic evaluation and administrative data interrogation were combined and summarised using a number of different impact frameworks that had been identified in Chapter 3, including the Payback framework.

This study has demonstrated how cancer research impact has been assessed in the past and has tested how impact analysis can be performed specifically for a cancer clinical trial. Evaluating the impact of the SCOT trial demonstrated its rapid and significant effects on new knowledge, future research, policy, clinical practice, and monetary savings for the health service. This assessment also allowed reflection on the pathway to these impacts occurring, as well as on how future trials could be designed to maximise impact. The study has highlighted challenges that currently exist to accessing real world data to investigate cancer trial impact. Further research to understand which impacts from clinical trials are meaningful to patients and trialists would be useful. More investment by funders and governments to support access to healthcare datasets that can be used to assess clinical practice change in response to trials would make impact assessments more straightforward in future.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Colleges/Schools: College of Medical Veterinary and Life Sciences > School of Cancer Sciences
• Funder's Name: Cancer Research UK (CRUK)
• Supervisor's Name: Jones, Professor Robert J. and Boyd, Dr. Kathleen A.
• Date of Award: 2021
• Depositing User: Theses Team
• Unique ID: glathesis:2021-82407
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 26 Aug 2021 08:54
• Last Modified: 28 Nov 2022 08:33
• Thesis DOI: 10.5525/gla.thesis.82407
• URI: https://theses.gla.ac.uk/id/eprint/82407

Related URLs

• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Article DOI
• Data DOI
• Data DOI