Vice President, Customer Success, Lokavant, Inc.

Todd Johnson

Due to more complex protocols and increasing data sources, clinical trials are generating triple the amount of data than the previous decade, with an anticipated rise over the next few years.

 This influx of study data isn’t necessarily a drawback; on the contrary, it is a great achievement for the clinical research industry, assuming the data is being leveraged by study sponsors and CROs to be proactive in their clinical trial development and operations. However, the rapid increase in data and disparate data sources has left the industry rather stunned. How do we accurately and efficiently manage this influx of critical study data?

In addition to the volume and diversity of study data, the quality is also of concern. In a study conducted by Harvard Business Review, 97% of critical data used in business decisions contains errors or is incomplete.

 Another study found that most executives are not confident enough in the accuracy of their data and analytics to take data-driven actions.

 More troubling, the United States, alone, wastes approximately $3.1 trillion on poor-quality data.

 Poor data quality equates to more than just monetary loss; for researchers it means critical time lost due to the need for manual rectification, all while attempting to stay on target with strict deadlines. Additionally, it can lead to insufficient data to assess the endpoints, resulting in delayed or complete absence of submission. But who pays the ultimate price for the increased cost and time spent? The patients relying on life-saving therapies, devices, and vaccines.

Lokavant and Halloran Consulting Group conducted a poll with leading biotech executives in clinical development to understand the state of clinical trial data quality, contributing factors, and how sponsors are rising to the challenges of data management. While data quality and management issues are certainly prevalent today, managing studies will only become more difficult in the future—68% of biotech executives said that they expect to spend more time on study oversight due to growing data complexity (see Figure 1 below).

 Lokavant and Halloran Consulting Group, 2022

While the clinical research industry spends a tremendous amount of effort ensuring the collection of high-quality clinical data, the same cannot be said for the operational data in a study. With a diverse set of tools and processes for data collection and analysis comes an abundance of imprecision. What causes this level of inaccuracy in operational data?

 The clinical research industry is filled with numerous data capture systems, unique processes and definitions, resulting in data management chaos. The same data element can vary between different studies, systems, and teams on the same study. The industry tends to overlook data governance, which involves alignment on data definitions to ensure high-quality operational data and compliance throughout the lifecycle of the study.

For example, "study start” is a term used by many organizations but has varying definitions. Some organizations may define this as the date when a patient is first screened in a study; others may define this as the date when the first site is activated. The same is true for many other study milestones, both clinical and non-clinical.

: Many clinical trial managers have not had the statistical training to manage the operational and clinical data influx. They are typically tasked with understanding the nuances of the data but lack critical skills required to interpret and analyze the data at scale.

With the rise of artificial intelligence (AI) in clinical operations, highly sophisticated algorithms are used to detect highly nuanced signals in large sets of study data. However, these technological advances are only valuable if they can be interpreted by the study operator within the appropriate context. In this nascent but rapidly developing field, the skillsets required to interpret these complex AI algorithms are often lacking, and clinical teams typically do not have the resources to keep up with the latest developments in AI.

Most biotech executives continue to heavily rely on dashboards and spreadsheets to review study oversight metrics (see Figure 2 below). While these tools offer many advantages,problems occur when spreadsheets and dashboards are used as business intelligence tools by many concurrent users. This can lead to siloes where the data are processed, analyzed, and altered outside a single source, creating multiple versions of truth.

If the available tools to aggregate and standardize data are not utilized, processes will take significantly more time to manage.Eliminating manual processes with a clinical intelligence platform will significantly reduce the exerted effort and improve data quality by maintaining a single space for users to easily access and analyze data.

The need for integration and automation is particularly crucial when data are being generated by many unique sources, a commonality across the clinical research industry. For example, hybrid or decentralized clinical trials (DCT) may generate data from remote monitoring technologies, patient self-reporting, site-based data collection, and many other sources. With numerous data sources enters the risk of multiple versions of the same data points. The ability to access study data from a single platform will reduce the need for significant reconciliation and allow for speedier, improved decision-making.

There is a cultural tendency in drug development to make decisions based on hearsay rather than data. For example, one myth that has circulated throughout the industry is that 20% of clinical trial sites will fail to enroll a single patient. This narrative has been widely accepted and relied upon to make critical decisions in study enrollment predictions. However, Tufts Center for the Study of Drug Development compiled an analysis of multiple clinical studies and concluded that only 11% of sites fail to enroll a single patient.

 Furthermore, this percentage can vary based on the therapeutic area or even the location of the study, providing more emphasis on leveraging contextually relevant data to drive critical study decisions.

Many organizations have the relevant historical study data and outcomes to forecast future performance more precisely. However, the effort required to systemically capture these insights is often too onerous, causing teams to default to heuristics, industry norms, or personal experience.

While culture and leadership are key enablers to better decision making, issues related to accessibility, skillsets, and quality are the primary roadblocks for biotech executives (see Figure 3 below). By realizing operational data problems exist, organizations can implement better practices to make improvements.

For most organizations, the root cause of these data issues will be a combination of poorly defined standards, a lack of integrated systems, and a skills gap with respect to advanced analytics. While organizations often look to solve each of these drivers separately, the most effective solution tends to be an integrated approach that addresses these drivers simultaneously.

Whether in-house or through technology partners, operators in clinical studies should adopt master data management principles in parallel to deploying an integrated platform that sets those principles into practice. This requires a data layer that allows study teams to continue their use of disparate systems, but harmonizes data across these systems, studies, and processes. While this represents an initial investment, it is more cost-effective to be proactive in generating high-quality, precise data than to later identify inaccuracies for resolution. Creatinga single source of truth will have an immediate impact of reducing reconciliation effort while ensuring better data quality.

Additionally, this high-quality data infrastructure allows organizations to unlock the value of more advanced data analytics, like artificial intelligence, to further augment data quality in increasingly large, complex clinical trial datasets. These analytics can support a wide-range of processes, from early risk detection and mitigation to risk-based quality management, to medical monitoring, to more effective study planning and forecasting—all of which lead to better data quality and improved study outcomes.

Since study teams may not have the expertise to interpret the output of these sophisticated algorithms, sponsors and CROs should consider platforms that are both intelligent and intuitive, allowing study teams to act within their familiar workflows rather than learning new statistical skillsets on a regular basis.

The challenges of data management and quality will continue to soar if the industry remains complacent. Ensuring high quality data in clinical research is paramount; the consequences of ill-informed decisions not only add to the ballooning costs in healthcare but could delay or preclude life-saving therapies to those who need them most.

This is the second in a series of articles where we explore how companies are harnessing and leveraging operational and clinical data to maintain quality and choice. Stay tuned for additional articles with useful insights on data-driven approaches in risk management, vendor oversight and portfolio management.

, Vice President, Customer Success, Lokavant, Inc.,

, Director, Sales and Marketing, Lokavant, Inc., and 

https://www.globenewswire.com/news-release/2021/01/12/2157143/0/en/Rising-Protocol-Design-Complexity-Is-Driving-Rapid-Growth-in-Clinical-Trial-Data-Volume-According-to-Tufts-Center-for-the-Study-of-Drug-Development.html

https://www.businesswire.com/news/home/20171107005745/en/Industry-Research-Shows-97-of-Companies-to-Increase-Use-of-Real-world-Patient-Data-for-More-Accurate-Decision-making

https://hbr.org/2017/09/only-3-of-companies-data-meets-basic-quality-standards

https://www.cio.com/article/236500/executives-still-mistrust-insights-from-data-and-analytics.html

https://hbr.org/2016/09/bad-data-costs-the-u-s-3-trillion-per-year

https://www.globenewswire.com/news-release/2013/01/15/1187761/0/en/New-Research-From-Tufts-Center-for-the-Study-of-Drug-Development-Characterizes-Effectiveness-and-Variability-of-Patient-Recruitment-and-Retention-Practices.html

Articles

Poll taken by leading biotech executives in clinical development addresses state of data quality and management.

The Future of Clinical Trials: Turning Data Chaos into Trial Intelligence

	Our industry has stepped up this year-in an uncharacteristic way. In an industry that has long been accused of being too conservative, lacking innovation, and being unwilling to change, we have proved the naysayers wrong. From fast-tracking the development of COVID-19 vaccines, to deftly welcoming telemedicine in ongoing clinical trials, to conducting virtual inspections, our response to this pandemic has been nothing short of amazing. Regulators have been furiously updating guidelines at record pace, sites have shifted resources and processes to accommodate patients, and of course patients continue to adapt to new remote and flexible solutions. 

	As Sponsors begin to shift from a reactive to a more proactive way of approaching the impact that the coronavirus has indelibly had on clinical trial conduct, they face several critical questions that they will need to have answers for.

	If Sponsors are not asking this question, they should be. In a recent virtual peer roundtable forum

, almost half (47%) of the attendees were not sure where to start. Not surprisingly, answers varied depending on what a Sponsor is trying to solve for. These same responders reported study delays (59%) and data collection (21%) as their main challenges during the pandemic. These are different types of problems with a variety of possible solutions.

	Whether the priority is patient access/engagement or data access/acquisition, the place Sponsors should start is always the same: strategic planning. This may sound like a cop-out, but leaders should be prepared to engage with their trusted stakeholders and outline an operational strategy that fits within the company’s vision. Conducting clinical trials quickly, while ensuring patient safety and data integrity is not a strategy-it is the result of one.

	As mentioned above, companies likely already have a good understanding around how the pandemic has impacted them. But they stumble when it comes to prioritizing what to address first and how. While a formal business case is likely not required for every scenario, understanding the components of one is a great place to start:

. From designing more patient-centric trials to obtaining more ownership of data acquisition. This list can and should be long.

. Solutions are not only technologies. Solutions can be process- and people-related.

. Next comes an evaluation of the benefits and risks of possible solutions for each of the identified priorities. Note that priorities may shift after this. 

. It is incredibly important to specifically establish both short- and long-term goals. These goals will have different assumptions and targets. One of the largest mistakes a company can make is assuming that the conduct and performance of a short-term pilot project will be the same as that of an enterprise one.

	You may notice that the above bullets sound like the formation of a strategy. And you would be right.  

	Gone are the days of focusing on minor risks that don’t move the needle if they indeed become issues. We have now experienced the Black Swan of risks with COVID-19 and need to prioritize rigorous risk identification and management processes during clinical trial planning. In the same industry forum mentioned above, 75% of the attendees indicated that they have plans to mitigate the impact of the pandemic. But what exactly are these plans?

	Perhaps the most important impact of the ICH E6 (R2) Guidance has had on the industry is informing companies that they have had woefully inadequate risk management processes. This “new” guidance (March 2018) expertly outlined the elements of a risk-based approach to quality management. And for the past two years, companies have been flailing in the wonderful world of key risk indicators (KRIs), quality tolerance limits (QTLs), and risk assessment and categorization tools (RACTs). Enter R3. 

	Without question, risk tolerance is shifting in the industry. New risks associated with decentralized clinical trials will become more visible and companies will need to identify and mitigate them accordingly. Well-positioned Sponsors are the ones that instill more provocative and strategic practices in their risk management processes. To become truly resilient, flexibility will be critical-and this begins with risk identification. 

	Including risks around data acquisition and data access are now elevated to “probable” likelihood with enormous impact, whereas before they were mainly categorized as unlikely to occur. Sponsors cannot be too provocative when it comes to the risk management process-this goes beyond the assumption that the COVID pandemic will recur. What about risk planning for viruses of a different type altogether-those that are technology-related? What happens if a computer virus knocks out your newly implemented remote monitoring solution? Or the site’s electronic health record (EHR)? Think and be curious. And of course, be flexible.

	Sponsors already have some information around this one-what have they observed over the last 3 months? Their reactive responses have served as harbingers for what their future processes will need to incorporate. 

	In addition to the risk management process changes discussed above, several additional process changes will include:

. Perhaps one of the main discussion points during the pandemic has been around patient centricity. Designing studies with the patient in mind (and with patient input) is increasingly being baked into protocol development processes. Mapping out the patient journey/experience during a trial provides Sponsors insight into what data can be collected remotely or not at all.

. For trials currently starting, Sponsors are hedging their bets when it comes to selecting sites for participation by including a diverse selection of sites - both geographically and by center type. Countries that have mitigated the pandemic well (Australia, New Zealand, Korea) are increasingly attractive. In addition, private clinics may be preferred instead of larger hospital settings (that may be prioritizing acute patient care vs. clinical trials).

. The entire on-site monitoring visit paradigm has been changing for quite some time-even well before the pandemic occurred. The proliferation of statistical monitoring and centralized monitoring of data has brought into question the effectiveness and utility of an on-site monitoring visit. This not only has ramifications on different technologies that can be used, but also on what (or if) partners are needed (both discussed below).

	It was not unintentional that we listed process before technology. A misguided decision many Sponsors make is choosing a technology solution without thoroughly considering the rationale of why it is even needed (or the resource and process implications it inherently brings). Having said that, it is no secret that those companies that have already implemented decentralized technologies (with a strategy) are better positioned because of them.

	Virtual trials have actually been around for quite some time (with Pfizer’s REMOTE trial leading the way in 2011). The resistance to faster adoption of technology-enabled trials is mainly due to self-imposed internal barriers rather than those of a regulatory nature. Other than Integrated Response Technology (IRT) and electronic data capture (EDC) solutions, clinical trial technologies were primarily viewed as optional-not even nice-to-have solutions. My, have times certainly changed. Technology-enabled trials are increasingly becoming the norm in a variety of different forms: eConsent, electronic Clinical Outcome Assessment(eCOA), centralized monitoring, remote monitoring, wearables and siteless trials. As vendors that offer these solutions continue to multiply, making the “right” choice becomes increasingly burdensome and intimidating. 

	So, the answer to what technology you need rests on the strategic questions you previously answered-and in many companies, technology needs are at the forefront of the conversation.   If different technologies were listed as possible solutions to your priorities, then there are additional considerations to address:

. This is where a business case makes perfect sense. Questions around “make vs. buy”, return on investment, and initial costs are projected and communicated to senior leadership. Sponsors should leave the heavy technology development to the experts-at the end of the day we are scientists, not software developers.

.A Sponsor’s structure must support its strategy. In other words, smaller companies may not have the internal expertise required to even identify an appropriate solution and make a business case to support its use. Let alone the resources required to support and ultimately maintain it. In addition, a Sponsor’s portfolio of studies may not be appropriate for a virtual/decentralized solution. This information is also a component of the business case.
		The infrastructure question is further supported by a recent survey

in Clinical Leader, in which smaller companies appeared more apprehensive around the conversion to virtualized trials. A total of 329 Sponsors responded that they agreed or disagreed with the question: “Virtual clinical trials will be a major component of my organization's clinical portfolio within two years”:

. Obviously, the mere act of buying a gym membership won’t make you healthy-you need to put in the work. This is no different when it comes to technology. Purchasing a solution doesn’t immediately make you data-driven or virtual. Change management and training cannot be underestimated when it comes to system implementation. Sponsors inevitably focus on the installation of the solution vs. trying to understand the implementation process.
		
		In order to obtain a ROI, companies need discipline to continue to use new technology rather than returning to their old ways of doing things. It's not just getting it in and implementing, it's continuing to drive usage until it's part of the fabric of the organization. Raising awareness, showcasing value, and reinforcing usage are all components of a robust change management strategy. Be patient – this is long journey and hopefully your long-term goals identified in your strategy support it.

	Talent is much more than the resources you have internally-talent also encompasses those partners you surround yourself with.

	Whether companies need to implement a new development strategy, update their risk management processes, shift (or reduce) monitoring resources or implement a new technology, one critical question remains: are these changes easier to make with partners or do Sponsors have the chops to go it alone? And should they?

	When asked how likely their CRO partners will be part of their future-state solution, industry professionals

	These results are telling in several ways. First, Sponsors are clearly considering new strategies. Whether these are based on perceived CRO inflexibility or taking more accountability for trial execution, Sponsors are positioning themselves to take more ownership of their data and processes. In addition to eschewing a CRO’s approach to treating each trial in a standard fashion, Sponsors are realizing that scaling resources is simply easier, and enables rapid decision-making without a CRO in the mix.  

	Next, the legacy CRO model is being challenged. Again, this is nothing new, but the pandemic showcased the limitations of the CRO. Two of the main revenue drivers for a CRO are on-site monitoring visits and project management time-both of which were rendered inadequate and useless during the pandemic. The ability to leverage data-either remotely or centrally for risk-based monitoring is changing the way Sponsors are approaching clinical trial execution. There are numerous benefits to owning your own data-from much leaner management oversight to more innovative, customized, patient-focused strategies to increased data quality.

	And data-driven approaches are proving to be increasingly cheaper in the long run. While there may be a significant initial investment in a technology, the long-term return on investment is proving to dwarf the cost of engaging a CRO. There are no more airplane and hotel pass-through costs, no more redundant project management fees, and on-site monitoring costs are replaced with substantially lower centralized monitoring and remote monitoring processes. Lastly, sites just prefer working with Sponsors-so this is a great opportunity for Sponsors to build more rapport with their investigators and become their Sponsor of choice.

	At the heart of these preparatory questions is the need to chart your future course. Are you contemplating Go? Or are you eager to GO! You can either wait around for best practices to emerge or you can boldly be the change you want to see in this industry. On your mark, get set…

 is a Consultant, both of Halloran Consulting Grou

		Halloran Virtual ClinOps Breakfast: Data Drive Development: Build vs. Outsourcing to CROs During COVID, Blueprint Medicines, 25 June 2020.	

		Fast Track To Flexible Clinical Trials, Clinical Leader, 7 May 2020.

Understanding how your strategy will be re-shaped post-COVID.

On your Mark…Get Set…Go?...or Go!

	As an industry, we produce data. Lots of it. It is inherent to what we do and clearly necessary for bringing innovative therapies to patients. While much of our time is dedicated to ensuring quality for the important safety and efficacy data in our submissions to regulatory agencies, the other data we indirectly produce-our operational data-can at times flounder (un-queried for quality) in our clinical systems (or ugh, myriads of spreadsheets). While “Big Data” is a great buzzword in management circles, small data (with big problems) never gets much attention.

	Our operational data is extremely important. It is a reflection of what we do and how well we do it. We can use it to evaluate our processes and initiate discussions for improvement. We can use it to forecast and extrapolate trends. We can use it to predict what will happen next and optimize expected outcomes. We can analyze it for hidden patterns that can help direct future decisions. So, what is preventing us from using this rich resource as other industries have? Every major sports organization has invested in analytics to better understand which players will succeed or which plays work best in a given situation, but how can we translate this capability to our own industry to resolve challenges with unproductive clinical sites? Apart from the pharmacokinetic/pharmacodynamic (PK/PD) modeling and simulation that our industry currently utilizes, we fail to prioritize operational data to make better decisions. The question is why? While there are likely many different reasons for this (i.e. lack of a data culture and/or strategy, technology limitations, lack of adequate skill sets, etc.), I would like to focus on a very basic, but important requirement: data quality.

	All the interesting things we can use our operational data for relies squarely on its quality. While analyzing data is exciting, unfortunately, cleaning it takes a great deal of time investment. A survey in the New York Times indicated that Data Scientists spend up to 80% of their time in data preparation.

We may be very familiar with the effort required to lock a clinical database, but we seem to continue to ignore that our other data require the same type of rigor to ensure it is able to be used for measurement and decision making. 

	So, as we look back on 2019 and look forward to the New Year, I would like to share some data resolutions that I will continue to evangelize throughout 2020 and beyond.

. This resolution is usually at the top of everyone’s list so let’s start here. Exercise is all about measurement and at the end of a long week, there is nothing more rewarding than practicing the Friday Afternoon Measurement (FAM) method. This is an exercise designed by the “Data Doc” and fellow statistician 

that provides a glimpse into how truly high the error rate is in your data.

The FAM method is actually quite simple and can be used on any spreadsheet (or validated data extract). The exercise is to assemble 10-15 critical data fields from the most recent 100 data records – and then simply count the number of records that are error-free. For example, your 100 recent records could be investigative sites and the critical data could include planned and actual visits dates (i.e. Qualification, activation, monitoring, closeout, etc.) and subject parameters (i.e. screened, consented, randomized, lost to follow-up, etc.). Even before completing the FAM method, how many error-free records do you estimate you actually have? If you said 3%, then you fit nicely with what Tom found in his study. And note, this does not even include records that have multiple errors (including missing data). Virtually every dataset is dirty to some extent and requires cleaning. 
		 

What better way to be more honest than to establish a single, trusted source of truth? Yes, it is easier than it sounds, and it is increasingly important as companies inherit new clinical systems. Master Data Management (MDM) initiatives are not at the top of every leader’s priority list. But honestly, they need to be if they are serious about creating a data culture. How many different answers have you received when you simply ask, “when was first patient in (FPI)?”, “how many patients are enrolled?” or “how many sites are currently in the trial?”? MDM programs not only consistently define each of these milestones (i.e. if FPI screened or randomized), but they also define exactly the source of this data. Think data dictionary and data standards. No, MDM is not the sexiest initiative to focus on, but it is incredibly important and should be initiated prior to any reporting initiative.
		 

Automate, automate, automate. Being human, I have no problem trying to eliminate my fellow humans from the data collection, data aggregation and data reporting process. In a nutshell, we are not very good at any of this. We not only lack the requisite focus and speed, but we insist on working in manual spreadsheets. If you have a “Metric and Reporting” task code in your time management system, you are well aware of how much time your employees are spending on manually creating dashboards/reports/trackers. It truly is incredible and the perfect ROI example for an automated solution.  
		 

. Digest only what is meaningful. We all love metrics and reports. But are all of them relevant and meaningful? To everyone? Building a governance around the data that is consumed at your company will enable you to begin eliminating the noise in your dashboards/reports and focus on the relevant signal. Understand there are different consumers of data, with different appetites. Question existing lengthy metric lists if there is no business reason or action expected. Interestingly, regulators continue to push the industry to focus on only what is important for patient safety and data integrity-ICH E6(R2) clearly drives companies to adopt a risk-based approach to quality management. This is a good thing. Not only does this put the patient first, but it also makes good data sense. From vendor oversight metrics and Key Performance Indicators to Key Risk Indicators and Quality Tolerance Limits-we seem to overindulge on what we measure, inevitably leaving unconsumed data on our plates. 
		 

. Become more data literate. Understand that there are different skill sets required to build an analytical capability. Spending time developing critical and analytical thinking skills is an important and necessary step in building a data culture. Data literacy is now required from everyone-not just the statisticians or data scientists that work with it routinely. Understand when to use a different measure of central tendency (median vs. mean). Remind yourself what a standard deviation is. Understand that there is natural variance in your data and this is acceptable (so that we can avoid designing our dashboards with narrow thresholds). Develop internal standards of performance. Collect. Clean. Access. Analyze. Benchmark. Question.
		
					 	

. Everyone loves a good performance benchmark. We are egotistical beings by nature and love to see how we compare to others. Companies pay big money to see how they are indeed performing compared to each other. While benchmarks are important to understand one’s performance, a mandatory first step is beginning with a standard. A good standard includes not only an expectation around performance, but also a clear definition and other supporting information. If there are clear industry standard definitions available, use them. Unique, internally developed metrics and definitions may be provocative, but you will have a tough time comparing yourself to other folks that have no idea what it is you are measuring. 

are incredible industry resources for process and metric standards. I highly recommend that any company in this industry learn more about the available resources these organizations have to offer.
		
		If you are not able to access any source of external industry standards, make sure your metric is at least standardized within your organization. Having different definitions (and different targets within the same function) leads to confusion – especially for new team members. Lastly, your standards should be centralized, versioned and documented within your organization, with clear expectations around review and updating.
		 

. First, look in the mirror. Now, I am really sorry to inform you of this, but your data is wrong. Exhale. Just like all datasets, the data that you are currently making decisions on is likely incorrect. It’s dirty and needs to be cleaned. Understanding that this is indeed the case is the first step to realizing how endemic this issue is in every organization. Data quality checks were covered in the first resolution, but the point here is that business and process owners should proactively be doing these checks so they can better understand the inherent biases that exist within their data. Understanding how to spot these biases is critical to creating processes that remove them.
		
		Do you ever wonder why your planned dates match your actual dates in your clinical trial management system (CTMS)? Despite what our optimistic selves may think, it is not because we are exceptionally good at forecasting. It may be because the planned dates were only completed after the actual data was known.
		
		Another example-take a mental note the next time you are included in any kind of metric/key performance indicator (KPI) review that involves colored indicators. How is it that the data behind the red indicators are always challenged and/or excluded? Why not challenge the green? This is confirmation bias at its finest. Humans like to confirm what they 

they know to be true. Mark Twain said it best: “It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.” Whether it is the overused (and erroneously applied) “80:20 rule” or anchoring on irrelevant data, critical thinking is required to ensure biases are not present in interpretation.
		 

it is being captured (insert your favorite polling joke here). Step back and take a long, hard look at your protocol and Case Report Forms (or CRF standards) and consider an external review. Collect only what is required. Just like ICH E6 (R2) has been the hot guideline for the past couple of years, I envision ICH E8 (R1) (and ICH E6 (R3)) will soon be on the tip of everyone’s tongues. Quality by design is not just pervasive in our guidelines, it makes good data sense. Don’t collect what you won’t use-especially when you have patients to consider. ICH E8 (R1) explicitly recommends patient input into study design.
		
		With the scope and complexity of protocols continuing to increase, understanding the data required to fuel these studies will become more burdensome. Virtual trials alleviate some of this burden by taking the trial to the patient-and data (coupled with technology) is the backbone that enables this to happen.
		
		On the operational side, understand how data is getting into your clinical systems (CTMS, TMF, etc.) and who is responsible for entering it. Understand the gaps and pain points of these processes since data quality is best addressed at point of entry.
		 

. More specifically, pass on the pie… charts. Data visualization is a science in itself-from understanding how our brains process images and color tones to simply understanding which visualization is the most appropriate for your messaging.
		
		For as limited as pie charts are for displaying quantitative data, they are ubiquitous in almost every PowerPoint presentation. I remember learning fractions by counting pie slices in grade school (and I do love a good pizza), but there must be an incredibly strong rationale for me to report on any data using a pie chart. They are the bright, shiny objects that serve to distract with limited purpose. And they take up valuable real estate on already information-packed dashboards and reports. To put it simply, pie charts are difficult for us to read (especially when there are >4 categories). Our brains are not good at differentiating differences when the pie slices are of similar size. And when the slices are not similar in size, you may be able to determine which slice is larger, but certainly not by how much. Ultimately, this leads to data labels being added, making the visualization itself not worth the precious space it is taking up.
		
		Don’t get me wrong, I have nothing against circular charts in general-donut charts are very effective for communicating certain messages. For those Apple Watch users out there, imagine if Apple had chosen to use the common pie chart (3 of them) instead of the donut charts used for your move/exercise/stand goals. How do you think your interface would be affected?  In a nutshell, we need to make data consumers think as little as possible so next time, use a bar chart or a table. 
		 

. While it is important to treat people the way you would like to be treated, it is also important to care for your data so other folks can consume it. And you should expect nothing less from them. As such, data democratization is an increasingly popular concept of simply providing enterprise-wide access to data. There is trust. There is transparency. And there is inherent quality built in. Data quality, like the data itself is an enterprise-wide and shared responsibility. Your quality warts are exposed as are your accomplishments. Data silos are demolished, and data literacy is fostered. This does not happen overnight-and culture plays an incredibly important role (as it does in every one of these resolutions). In this era of fake news and alternative facts, the importance of democratizing data access and data quality is paramount. The great quality management guru W. Edwards Deming once said, “Without data, you’re just another person with an opinion.” Precisely.

	These resolutions are in no particular order as they are all important. And some are certainly more labor intensive than others. A critical underlying component is having a culture that understands the importance of data and fosters supportive behaviors around it. Just like having a partner to exercise with increases the likelihood that you will go to the gym, having a leadership team (equipped with a data strategy) that evangelizes data as an important asset is the most important element for analytical success. 

	The question is not really if you will implement these, it is when you will need to. In 2020, the life sciences industry will inevitably see new growth in virtual trials, wearable technology, personalized medicine, artificial intelligence and machine learning – and GOOD data is the cog that makes all of this possible. Unfortunately, BAD data can go through these motions, too, and can get you to the wrong conclusion. Other than your people, data is the most valuable asset your company will produce. This year does not have to be the same as previous years when it comes to instilling a new appreciation and understanding of data. You can start today. 

 is a Senior Consultant at Halloran Consulting Group, Inc.

	1. For Big-Data Scientists, ‘Janitor Work’ Is Key Hurdle to Insights, Aug 17 2014.

	2. Only 3% of Companies’ Data Meets Basic Quality Standards, T Nagle, T Redman, D Sammon, Harvard Business Review, Sep 11, 2017.

While “Big Data” is a great buzzword in management circles, small data (with big problems) never gets much attention.

Todd Johnson

Articles

The Future of Clinical Trials: Turning Data Chaos into Trial Intelligence

On your Mark…Get Set…Go?...or Go!

My 2020 Data Resolutions