Data Science and Statistics, University of Michigan

Gwenyth Jones

Secondary use of individual patient data (IPD) generated through a randomized clinical trial (RCT) represents some of the highest-quality research data available. Clinical trial data is collected with the patient’s consent from trials designed with clearly defined endpoints derived from objective assessments. Effective reuse of these data has the potential to transform the clinical research process, improve trial design and execution, and respects the patients who donate their time and their data as part of the clinical development process.

The transformative potential of effective data reuse has resulted in calls from groups including the World Health Organization,

 and patient advocacy groups for sponsors to openly share and reuse clinical trial data. However, achieving widespread use of clinical trial IPD also requires data contributed by trial sponsors to retain as much research utility as possible. A trial sponsor’s data sharing policies determine what data and supporting documentation to share, where and how that data will be shared, and what data transformations are needed (e.g., to protect patient privacy or intellectual property). 

The policies applied to secondary use data contribution have a direct impact on determinants of research utility. These determinants include data comprehensiveness and completeness;

 the supporting documentation and contextual information provided (e.g., transformation reports and data dictionaries); and the metadata to aid end-users in determining whether the contributed data will be fit for purpose for the intended research use.

Across trial sponsors, there is significant variability in data sharing policies and applied data protection methodologies.

 This variability creates substantial challenges for Data Sharing Platforms (DSP) and their end-users, especially when research use cases require pooling or integrating IPD from multiple sponsors. This, therefore, hinders efforts to make the shared data “FAIR” (Findable, Accessible, Interoperable, Reusable).

Two dimensions influence how well the data sharing ecosystem serves the needs of the research community. The first of these dimensions is access. Access can be thought of as the number of trials available for secondary use from a given sponsor and on a given DSP in conjunction with the DSP policies that govern research access.

The second dimension and focus of this paper is data utility, or how well contributed data meets the needs of researchers. While this paper focuses on the needs of the researcher, the results are also important for data contributors. Data contributing organizations are often consumers of data. This is particularly true for biopharma sponsors, many of whom have made significant investments in internal data sharing platforms bringing together a sponsor’s internal data and data from external data sources.

 Besides being a potential end-user beneficiary of a well-functioning data sharing ecosystem, data contributors may also benefit by better meeting the research community's needs and thus ensuring their investment in data sharing infrastructure delivers a maximum return to the community.

Understand key factors of research utility for users of secondary-use clinical trial datasets containing IPD.

Determine the relative value to researchers of various types of IPD datasets, supporting documents, and metadata as a basis for future secondary use data standards designed to maximize the research value of clinical trial data.

Identify gaps in researcher or data contributor knowledge that present opportunities for community education.

The research scope included surveying information and elements that may be available to end-user researchers accessing or using secondary IPD clinical trial data irrespective of therapeutic area and specific research objective. These areas included:

: What patient-level datasets are needed by researchers?

: What trial-specific supporting documents do researchers need to enable secondary use and maximize research value?

: What information about the trial is needed to provide sufficient context for dataset access and use?

: When is the provision of selected supporting documents and metadata needed to facilitate process efficiency for both data contributors and researchers?

Other areas that may impact research use and utility were out of scope. Specific research use cases may impact the information and elements required by IPD end-users; however, exploring the nuances of specific use cases was out of scope. The authors did not examine the research impact of dataset or documentation content or completeness. The area of patient consent and/or legal basis for data contributions was not included because, while relevant to data contributors,

 it was deemed to be outside the general scope of end-user considerations. The Data Protection methodology applied to the processing of datasets and documentation is an important determinant of data utility

but was not included in this research because the process steps are generally not evident to the contributed data end-user. Information about the result of the data protection process (e.g., a variable-level transformation report) is included in the research scope.

Survey conceptualization, distribution, and data collection

The survey design goal was to determine which elements of data contribution are important to IPD end-users. The survey was designed to accommodate a range of Data Sharing Platform (DSP) access models, ranging from controlled, gated, or organization-only (internal) access to open data sharing platforms. The target audience was DSP end-users who requested, downloaded, accessed, or used clinical trial datasets containing IPD for secondary research purposes.

The survey content was derived by collating the datasets, documentation, and collected metadata for secondary clinical trial contributions across multiple DSPs. Survey questions were divided into three areas:

Demographics: The respondent’s role, experience level, and general secondary use case(s) for contributed clinical trial data containing IPD.

Contributed Trial Content: What clinical trial datasets, supporting documentation, and metadata are needed to support the respondent’s intended use(s)?

Recent Experience: Information about the respondent’s most recent DSP interaction

The survey responses were returned anonymously. However, respondents had the option to provide contact information, and, in some cases, verbatim responses included information that could identify the respondent and/or their organization.

Survey distribution represented both academia and industry and encompassed the spectrum of DSP access models. The survey was distributed to relevant research communities through multiple DSPs, biopharma companies, and non-profits. Due to the breadth of distribution, which may include organizations or channels the authors were not aware of, a list of distributing organizations is not provided. In some cases, organizations were provided with an organization-specific survey version to allow for subsequent comparison of their respondent population to overall responses. All distributed survey versions were identical. Other distribution included CRDSA newsletters, social posts (LinkedIn), and conference mentions. 

Prior to analyses, the data from all organizational surveys were merged into a single master dataset. All respondent identifiers were anonymized by removing or redacting identifying information, including within verbatim responses and email addresses (if provided).

All analyses were carried out using Google Data Studio.

 The data variables were grouped into nominal categories (categorical variables). All responses were summarized with descriptive statistics (number and percentage of responses to the different nominal categories), and no statistical hypothesis testing was carried out.

Demographic sub-groups with similar characteristics were consolidated into categories to enable the interpretation of responses. Organization types were consolidated into three categories for response analysis:

 (Survey response of Academic Research Organization, Non-Profit or Foundation, Individual Researcher, Scientific Interest Group)

 (Survey response of CRO, Service Partner, Technology Vendor)

To facilitate comparison by organization type when assessing responses to the survey questions, the results for organization types with low response frequencies were not presented when the sub-group had consistently similar response characteristics to another sub-group and are only presented when different in response. The computed total frequencies, however, were inclusive of all sub-groups.

The secondary use data experience level was also grouped into three categories and relative frequencies were computed for the different organization types.

Survey responders who identified themselves as “Power Users” or “Very Experienced” were combined into a single group.

Respondents were asked which DSPs they use. Respondents could select or enter multiple DSPs. To allow for analysis of the responses to all data sharing platform engagements (single or multiple), responses were grouped into five ordered categories:

The frequency of engagement was then computed and compared across the different organization types.

Further, response analysis of data sharing platform usage (single or multiple) was done by first applying SAFE data platform categorization per Bamford, Arbuckle et al. “Sharing Anonymized and Functionally Effective (SAFE) Data Standard”

 and then computing the frequencies of responses for the different organization types. See Appendix A for the list of Data Sharing Platforms (inclusive of free text responses) included in the analysis (inclusive of free text responses).

The purpose of data request/usage such as peer reviewed journal publication on new knowledge or insights, informing clinical trial design, generating safety or adverse events reports and to supporting investigational new drug applications (IND) were categorized into three groups: Academic research/publications, internal organization use, and regulatory submission. To allow for further descriptive analysis, responses were categorized as:

The frequency of usage (number and percentage) were computed for each organization type.

The number of studies requested (expressed in percentages) and the relative usability frequency or yield, as well as the reason for non-use and their frequencies, were also calculated. The usage yield was then grouped or categorized by respondent experience level to allow for comparative analysis.

Dataset types such as ADaM, SDTM and data model description documents such as data dictionaries as well as study supporting documents (e.g study protocol) are key components of secondary use data requests. Supporting documents provide context to the supplied IPD datasets and aid the researcher in understanding trial design and execution factors that may impact their research use.

Respondents were asked to assess datasets and document types that may be supplied by data contributors and assign each to four ordered categories: 

 and the percentages were calculated for the responses for their importance for secondary use. For analysis purposes, 

The authors are aware that certain sponsors furnish a study-specific data transformation report documenting the transformations at a variable level that took place during the data contribution anonymization process. The authors understand that this type of data transformation report is not routinely or uniformly provided by data contributors. To evaluate the value to the research community, Respondents were asked to assess the importance of this type of report on a 1 to 5 scale from 

Respondents were also asked to evaluate the importance of having key metadata and study parameters prior to download or access. For analysis, responses to the type of metadata was grouped into three categories:

Their frequencies expressed in percentages were then computed for each type of metadata.

The timing of access to certain supporting documents and other relevant information can provide important context prior to data request, access, or download. Based on the author's experience, key documents or information that may assist in determining suitability for a study for a particular research question or need can include a redacted study protocol and information about key data redactions such as adverse events, demographics, and laboratory values.

For the purpose of this research, survey questions on the timing of access to these supporting documents included redacted study protocol, adverse events, demographics and laboratory values.

Response analysis of the timing of redacted study protocol access and relative importance was done by calculating the response frequencies for the different timings per the following categories: 

Response Analysis of whether data redaction information was needed and, if so when data redaction information should be made available to the researcher was calculated for each redaction type.

The analysis results were summarized in tabular formats. All survey data, redacted for respondent privacy as noted above, is available in the supplemental tables referenced in Exhibit A.

The survey was developed by a diverse multi-stakeholder group of subject matter experts, including individuals with data contribution expertise, research use experience, and Data Sharing Platform governance responsibilities. The survey questions were further informed by collated data contribution information from multiple DSPs.16

The results are an artifact of survey distribution, which was limited by the contacts and reach available to the authors. Therefore, the results may not reflect the full diversity of the clinical research ecosystem, and the respondent distribution may under-represent certain organization types. In particular, the authors are aware that there is a gap in small biopharma representation due to the relative size of the Clinical Research Data Sharing Alliance’s (CRDSA) biopharma member companies.

Survey response rates varied by distribution source and organization type, which may serve to bias responses toward those most engaged in the data-sharing ecosystem. Therefore, one-time or limited-use researchers may be under-represented.

The survey focused on some of the technical elements of data sharing (documents and datasets, timing of document provision, etc.). The authors recognize there are many non-technical aspects of the data sharing ecosystem that have a significant impact on the researcher experience. These include, but aren’t limited to, platform and data contributor access policies; intellectual property and competitive considerations; and data contribution resource investment.

A total of 104 respondents completed the survey. Respondents were grouped by organization type and are presented in Table 1.

The number of Service or Technology Vendor responses was too low to draw conclusions reliably representative of the category. In almost all cases, the Service or Technology Vendor responses were not meaningfully different from those provided by biopharma respondents. Therefore, the presentation of comparison results focuses on Academic/Non-Profit and biopharma responses, noting where Service/Technology responses differ from the biopharma responses. “All” is inclusive of total responses from the three categories, including Service or Technology Vendor responses.

Biopharma experience levels were almost evenly distributed (Figure 1). Biopharma respondents were more likely to be in the 

 Category (32.7%, n = 16 of 49) when compared to Academic / Non-Profit (23.3%, 10 of 43). Conversely, Academic respondents were more likely to be 

 (48.8%, 21 of 43), with a lower percentage of 

 (27.9%, 12 of 43) when compared to biopharma (34.7%, 17 of 49).

 Data sharing platform engagement frequency

95.2% of overall respondents (n = 99 of 104) engaged with data sharing platforms one or more times per year (Table 2). There were no meaningful frequency differences between organization types.

On platform usage, Respondents noted 10 unique DSPs, with an average of just under two platforms (1.93) per respondent. The responses were categorized by applying the SAFE data categories and are presented in Table 3.

 SAFE data platform categorization frequency

 category (1) includes data warehouses/data marts internal to organizations. External data sharing platforms are categorized as 

 platforms. Compared to Academic / Non-Profit respondents, biopharma respondents are more likely to use internal data platforms (20% versus 6%, respectively). Service and Technology vendor responses (included in “All”) are not summarized separately in the table due to the limited number of respondents; however, those responses did diverge from both biopharma and Academic / Non-Profit, with 33% (4 of 12) of mentioned platforms in the 

The results of respondents purpose (secondary use) of data request per organization type is shown in Figure 2 below:

Service and Technology Vendor responses showed characteristics of both other categories, with a 92% response rate for Academic Research use (approaching Academic / Non-Profit response frequency) and a 42% Regulatory Submission response rate (similar to biopharma response frequency).

Results of respondents usage yield per experience level is presented in table 4 below.

As the respondent experience level decreases, the usable study yield decreases, with 100% yield decreasing by 29.3% for One-Time Users compared to the most experienced respondent group.

Results of responses were consistent across organization types and respondent experience levels and are considered as a pool group in Tables 5 and 6.

The results of responses on the importance variable-Level data transformation report were also consistent across organization type, and respondent experience levels and are considered as a pool group in Figure 3.

Importance of a variable-level data transformation report

Results of the importance of key metadata required by researchers prior to access to study data are presented in Table 7 below:

Articles

Study seeks to understand how different forms of data meet the needs of researchers. 

Gwenyth Jones

Articles

Establishing a Basis for Secondary Use Standards for Clinical Trials