Russian regulators have reportedly implemented the much awaited changes to the nation's regulatory process for medical devices, and the revised regulations took effect at the start of January. This follows a highly positive recent report about the general state of clinical research in Russia in a leading US peer-reviewed cardiac journal. 

The medical device regulator, Rozsdravnadzor (Federal Service on Surveillance in Healthcare and Social Development of Russian Federation), has devised new procedures for testing sample import permits, as well as expert device reviews and clinical testing requirements, according to Stewart Eisenhart, who covers medical device regulatory affairs for the Emergo Group. 

A significant change in the revised regulations concerns the larger role "expertise centers" will play in the Russian medical device registration process. These centers will determine whether registrants must conduct clinical trials in Russia for their devices, as well as whether devices under review are safe enough to undergo clinical testing in Russian healthcare facilities, he explained in an article posted on the Emergo website on January 15. They will evaluate registration dossiers for quality, safety, and efficiency, which will influence the regulators' final decision about whether to approve a registration application, so foreign companies seeking market authorization in Russia should prepare to deal with multiple entities, not just Roszdravnadzor. 

"With a revised regulatory process now in place, medical device registration in Russia still remains a complex endeavor. But now that we have a more detailed playbook, so to speak, of what to expect in terms of Roszdravnadzor requirements, medical device companies may ultimately benefit in terms of a more predictable (if not easy) path to market in Russia," noted Eisenhart. 

For the first time, Russian regulators have officially introduced requirements for authorized representation. They define an "authorized representative" as a legal entity registered in the Russian Federation and representing a medical device manufacturer in regulatory issues, including conformity assessments and registration. 

Testing results will still play a large role in Russia's medical device registration process, he stated. The new regulatory system adds extra testing requirements for measuring devices, but it remains unclear whether regulators will require clinical testing for all risk classes or whether they will accept clinical test results from internationally accredited laboratories. 

Medical device registration certificates obtained before January 1, 2013 will remain valid until their expiration dates, according to Eisenhart. Registration certificates for devices approved prior to the new regulations that have no expiration date remain valid until January 1, 2014, at which point companies will have to submit official statements to Roszdravnadzor requesting replacement of registration certificates. These manufacturers will not have to re-register their devices, he believes. 

Meanwhile, Anthony N. DeMaria, MD, Editor-in-Chief of the 

Journal of the American College of Cardiology

, has issued an upbeat report about clinical research in Russia. "The implications of my Russian experience were particularly impressive for clinical research," he wrote in an editorial in the December issue. "The channeling of clinical volume, such as into the Moscow City Center, provides an excellent opportunity for clinical investigation."  

DeMaria notes that large, impactful trials now draw up to 80% of patients from countries outside of the industrialized world, and he thinks the trend toward an increasing amount of clinical research being performed in emerging markets will continue. The incremental benefit of many therapies is often modest enough that large sample sizes are required to determine benefit, and many study sites are necessary, he maintains. Also, the cost of research is much less in developing countries, especially since enrollment is usually more rapid and thus the duration of the trial is shortened, and the regulatory environment outside the United States is often less demanding. 

Based on his experiences in Moscow, it is clear that centers with contemporary technology and expertise do exist. Cardiovascular patients in these centers are managed in a standard fashion according to existing guidelines by knowledgeable and skilled physicians using up-to-date pharmaceuticals, supplies, and equipment, and patients can be enrolled in studies with the expectation that the results observed can be extrapolated to a general population, DeMaria concluded. 

Articles

News

Russian regulators have reportedly implemented the much awaited changes to the nation's regulatory process for medical devices, and the revised regulations took effect at the start of January.

Regulatory Landscape Changes in Russia

As I write this column, we're midway through the NFL playoff season, when the haves root their competitive, winning teams through each sudden death round, while the have nots (losers like those of us from Chicago and Philadelphia) are anxiously following the search for new coaches and innovative plans to rebuild a team to make it playoff worthy in the shortest period of time. One could call this latter activity translational sports management, where ambition, vision, and hope are ideally converted into future success on the playing field—or so we can dream when training camp opens again anyway. Coincidentally, this season of hope also just precedes the salad days of spring training in baseball, where everyone's a winner until the actual games have to be played. 

Clinical development has the somewhat analogous concept of translational research, which, in the simplest sense, means steps taken to bring new treatments from the academic world to real medical practice—from bench to bedside. The National Cancer Institute's Dictionary of Cancer Terms defines "bench to bedside" as "a term used to describe the process by which the results of research done in the laboratory are directly used to develop new ways to treat patients." The definition of "translational research" is almost identical, though in that case the focus is described as developing new ways to "diagnose and treat disease."  

No matter what we call it, the need is obvious. It is estimated that less than 1% of drug compounds discovered by researchers that are deemed worthy of initial testing ever make it to the pharmacy—pretty much long shot odds. And getting to market typically costs over a billion dollars over 15 or more years—a span of time that can affect many patient lifetimes.  

Fortunately, there are many initiatives already underway that fit under the promising big tent of global translational research. Nearly a decade ago, the US National Institutes of Health's NIH Roadmap for Medical Research outlined a series of programs that provided an early boost to translational research, in the hope of addressing roadblocks and knowledge gaps. The NIH Roadmap was broad and highly ambitious, involving programs ranging from identifying new pathways to discovery in such areas as bioinformatics, molecular libraries, and nanomedicine, to reengineering the clinical research enterprise. The reengineering research program involved goals to improve regulatory processes, enable national electronic clinical trials and networks, and introduce new technologies for assessing disease. A third roadmap area involved activities to develop research teams of the future, by establishing advanced training programs; building collaborative partnerships involving academics, physicians, and patient communities; and developing public/private partnerships, a theme that continues to persist today.  

				Although references to the original NIH Roadmap are not as easy to find anymore, the NIH is now advancing similar goals through the NIH Common Fund and National Center for Advancing Translational Sciences (NCATS). The mission is "to catalyze the generation of innovative methods and technologies that will enhance the development, testing, and implementation of diagnostics and therapeutics across a wide range of human diseases and conditions and make translational science more efficient, less expensive, and less risky." 			

Many similar activities were further moved along by the FDA through its Critical Path Initiative, which, among other things, led to public/private partnerships like the Critical Path Institute and the Clinical Trials Transformation Initiative. More recently, TransCelerate Biopharma's consortium of major pharmaceutical companies was established to sponsor projects with the intent of improving the efficiency of research, advancing innovation, and solving common R&D challenges and obstacles in a cooperative, precompetitive environment. 

Meanwhile, in Europe, the Innovative Medicines Initiative (IMI), a public/private partnership between the European Union and the European Federation of Pharmaceutical Industries and Associations, was created to "speed the development of better and safer medicines for patients," identifying translational research as one of its cornerstone areas for investment. Thus IMI's European Translational Information and KM Services project (eTRIKS) intends to provide a knowledge management technology platform to support IMI projects that will analyze and compare diverse collections of shared data for multiple studies and treatments. eTRIKS is intended as a resource for collaborative research, knowledge discovery, and knowledge management using data from clinical studies and pre-clinical studies, imaging data, genomics, proteomics, and data sources for biomarker discovery and validation. 

Other comparable national initiatives abound elsewhere around the world, such as the Translational Research Informatics Center in Japan.  

What kinds of projects are covered under the umbrella of translational research? Well, for IMI eTRIKS, translational research studies may involve working with clinical data and collected biological specimens from laboratory animals and study volunteers to generate biological profiles, and support simulations and predictive model development or validation (such as by helping to identify more effective animal models to use during pre-clinical testing to better predict how humans will react). Such projects can help identify new opportunities for drug development, or provide greater insight into potential safety issues that can help researchers identify more promising drug candidates, rapidly eliminate those likely to fail, and predict what populations are most likely to respond favorably during clinical testing— knowledge that ideally can be applied to help new drugs become more rapidly available to patients.  

So in essence, you get lots of different types of data together all in one place, bang the data around, and see what you can learn. Then—and this is the critical translational part—figure out how to put what you learn into practice so that others can apply it to make a real difference for patients. 

So translational research is a hot topic, and it seems as though nearly everyone everywhere is trying to solve similar problems. But alas, mostly on their own. Ironically, one possible observation is not that so many organizations are involved in activities related to translational medicine and transformational research, but rather that there are so many separate, similar initiatives operating concurrently around the world. It's difficult just to keep track of them all, much less coordinate them to avoid duplication of effort and maximize sharing of ideas.  

Oddly, the Internet, that ubiquitous gateway to information and big data, is as much a culprit as enabler in this regard. Search engines are so prolific and indiscriminate that it's nearly impossible to chase every lead. You need to make choices, and often feel that rather than trying to find an existing wheel that will fit the specific research vehicle you're building in an unmarked junkyard of spare parts, it's simply easier just to reinvent one yourself.  

This overlapping or duplication of effort can occur on many levels. For example, most scientific publications list references of prior work that may have informed the current research project—the question is how deeply and comprehensive this list should be or can be if the author is ever going to get around to saying anything new at all. And given the complexity of research projects in life sciences, it's pretty easy to rationalize that nobody else could possibly be working on the exact same thing anyway, so why bother reaching out too far?  

Meanwhile governments and foundations fund myriad research projects, which often develop tools, lots of tools, that may support dissertations and publications but are then left along the wayside while others are exploring similar problems over and over again. Perhaps too many tools are being created, making for a complicated landscape of too many choices when someone is looking for help solving their own problem. Sure, funding research may lead to great new discoveries, but we need more alignment among global research projects in academia, government, and industry as well. Surely there are cases when the game is more likely to be won when we all learn to put our resources into the same team. 

One possible avenue toward improved collaboration among translational researchers might be creating a global master registry or clearinghouse to tie together the various projects and information resources and connect researchers in the same way that we connect to other kindred spirits via social networks. Researchers exploring their topics could be informed of possible relationships—similar to how iTunes and 

 informs us, when we're exploring a potential purchase, that people who have viewed this item in the past also purchased something else. We all need to get on the same page if we have hopes of solving common problems in a common manner quickly enough to truly reduce the gap between bench and bedside. 

In clinical development we work in a field that stems from academic and scientific research, where careers are made through individual invention and discovery. It's not easy to reach across the aisle to others who may be working on similar problems, much less those across multiple time zones and cultures who speak different languages yet have a common need to improve human health. But to make great strides in translational medicine, to reduce the nearly two decades it currently takes to go from bench to bedside, we need to, above all, learn from each other, work together, and transform competition into cooperation. Or, in the oft-quoted words of Benjamin Franklin, "We must all hang together, or assuredly we shall all hang separately." 

 is Chief Technology Officer for the Clinical Data Interchange Standards Consortium (CDISC). He resides near Chicago, IL, and can be reached at 

CTI Insights

To reduce the time from bench to bedside we need to transform competition into cooperation.

Cutting Our Losses in Translation

ACT Cover

While there is no single correct way to develop process compliance controls to meet federal clinical trials billing regulations around Medicare, standardization of the entire billing process is key. Establishing standards around a comprehensive clinical trial billing compliance program will help mitigate billing non-compliance risks. The risks of not complying with federal clinical trial billing regulations can lead to research suspension, fines, and/or the imposition of corporate integrity agreements.  

There have been numerous federal investigations and settlements involving improper clinical trials billing. Perhaps one of the most notable investigations involved Rush University Medical Center. In 2003, Rush reviewed its clinical research operations and uncovered a number of errors in which Medicare was improperly billed for research services as routine costs. 

Effective July 9, 2007, Clinical Trial Policy National Coverage Determination (NCD) from CMS, says Medicare covers the routine costs of qualifying clinical trials, as well as reasonable and necessary items and services used to diagnose and treat complications arising from participation in all clinical trials. Previously, Medicare did not cover patient care costs associated with enrollment in a clinical trial. The NCD for Routine Costs in Clinical Trials (310.1) is now the standard by which commercial payers base clinical research coverage decisions. 

Routine clinical trial costs include all items and services that would normally occur as part of the patient's care outside of a clinical trial. Costs associated with the prevention, diagnosis, and/or treatment of complications arising from participation in clinical trials are also covered. Standard Medicare billing rules apply to items deemed routine costs under the NCD. If Medicare covers the costs of items and services outside of the clinical trial, then they are covered during the clinical trial. 

A clinical trial must meet the following requirements under the NCD to receive Medicare coverage for routine costs: 					

The subject or purpose of the trial must be the evaluation of an item or service that falls within a Medicare benefit category and is not statutorily excluded from coverage.

The trial must not be designed exclusively to test toxicity or disease pathophysiology. It must have a therapeutic intent.

Trials of therapeutic interventions must enroll patients with diagnosed disease rather than healthy volunteers. Trials of diagnostic interventions may enroll healthy patients in order to have a proper control group.

Exhibiting these three criteria, however, does not automatically qualify a clinical trial for Medicare coverage of routine costs. There are seven desirable characteristics that clinical trials must possess. Some clinical trials are automatically qualified as they are presumed to meet the following characteristics: 					

The principal purpose of the trial is to test whether the intervention potentially improves the participants' health outcomes.

The trial is well-supported by available scientific and medical information or it is intended to clarify or establish the health outcomes of interventions already in common clinical use.

The trial does not unjustifiably duplicate existing studies.

The trial design is appropriate to answer the research question.

The trial is sponsored by a credible organization or individual capable of executing the proposed trial successfully.

The trial is in compliance with federal regulations relating to the protection of human subjects.

All aspects of the trial are conducted according to the appropriate standards of scientific integrity.

Under the NCD, many clinical trials' study services can be considered routine costs, but the definition of routine costs requires careful examination of the study protocol and the sponsor's contract to determine which items and services are payable by the sponsor. These items and services are not covered by Medicare. The financial disclosure language in the informed consent form also must be examined carefully. Items and services disclosed to the research participant as being free or paid for by the sponsor or third-party payer are not billable to Medicare.  

Clinical trials billing compliance seems like a simple concept: Do not bill Medicare for services being paid by a sponsor and/or third-party payer, and do not bill Medicare for services that do not meet the requirements under the Clinical Trials NCD. Many stakeholders question whether the Medicare rules accomplish their goals, and in fact, the CMS has proposed a number of changes in an attempt to clarify gray areas surrounding the rules. 

A comprehensive clinical trial billing compliance program can help organizations establish standards to meet regulatory requirements and provide sustainable organizational consistency. 

Jim Moran, Erika Stevens and Julie Statzel, Ernst & Young LLP 

*The views herein are those of the authors and do not necessarily reflect the views of Ernst & Young LLP.

While there is no single correct way to develop process compliance controls to meet federal clinical trials billing regulations around Medicare, standardization of the entire billing process is key.

Medicare and the NCD

Until relatively recently, taking your company's clinical research into the realm of electronic data capture (EDC) meant going with a full service vendor with a price tag generally starting at $175,000 to $200,000 for a 250 patient, 50 site trial over 24 to 36 months, and as high as your project was complex. If you wanted to conduct smaller scale research these were often just not economically feasible-the same pricing as just mentioned might be provided for a 50 patient, five site trial.

	Of course, the option has been available from some of the larger EDC companies to "do it yourself" (DIY) in an attempt to decrease costs, but these were often only less expensive if you were doing large volumes of research and could afford to have skilled "build" specialists trained and on staff. The net result is that smaller companies either remained in a paper world with all its attendant woes, or were compelled by cost-containment needs to select markedly sub-par EDC systems almost as problematic as the old paper model.

	Of late however, perhaps over the past three to four years, viable DIY EDC options have begun to emerge onto the marketplace that make what has become a commonplace technology in larger organizations more accessible economically to smaller enterprise. Finding, assessing, and selecting DIY EDC though, is an activity that is by no means as simple as it might at first appear. Having lived through this exact process very recently, this article seeks to share experiences in the hope they may prove useful to others in similar positions.

	The company for whom I was evaluating systems, a small venture capital backed medical device company, had previously conducted a number of studies on paper using as a backend a well-known and relatively inexpensive CDMS. The largest and most recent of the studies conducted required a considerable amount of data cleaning after the enrollment period closed-largely because data errors had been introduced due to the nature of a dynamic non-point-of-capture-checked data acquisition. The burden placed on the company in cleaning up this rather complex trial dataset was very significant, and contributed in no small part to the decision to investigate DIY EDC systems.

	Given the upcoming projects, which varied from moderate to quite small in nature, full-service EDC was not seen to be a viable option. I was requested, given my background in EDC/CTMS system design and implementation, to source, evaluate, and recommend a DIY option. Cost was not a primary consideration, but neither was it immaterial, as one might expect in a small organization.

	With years of experience and more than a few systems under my belt, I foresaw little difficulty in conducting this project-get a few demos, jot down a few notes, make a recommendation, and sign contracts. Much to my surprise and chagrin, my assumptions were proved wrong almost immediately.

	It became obvious, fairly quickly, that problems with the initial methodology as conceived were legion. These included:

		The simple task of developing a vendor list was not easy.

		Scheduling and sitting through system demonstrations is a time consuming laborious process.

		The disparity between systems and capabilities makes non-systematic comparison inherently an "apples to oranges" exercise and hence invalid.

		The variation in pricing models made cross-comparisons difficult (this may be a deliberate ploy on the part of the vendors).

		Specialist knowledge is required to get past minutiae and understand true capabilities (or lack thereof).

		Sales reps have a habit of showing you what is great about their system and glossing over weaknesses-this can be hard to detect without an understanding of what particular system characteristics actually mean in the real world, and especially so outside of a structured demonstration.

	Many, perhaps even all, of the issues listed above exist for a company of any size. In a small company however, the problems, or their impact, become exacerbated, mostly due to far more limited resources in terms of personnel, diversity of expertise/experience, and time.

	Developing a list of candidates for assessment was addressed through looking at resources such as ClinPage (Who's in the news? Who are the advertisers?), lists of conference exhibitors (SCDM, Partnerships in Clinical trials, DIA etc.) and most usefully the 

 Annual Buyer's Guide. Using search engines such as Google, Yahoo, Bing, and others proved less than useful with an awful lot of CROs and non-clinical research organizations showing up in results.

	Other online resources include the various groups on LinkedIn that focus on EDC and CTMS-like ClinPage, these are useful for uncovering vendors that might be suitable-caution should be exercised with the postings/discussion though, as many contributors are vendor employees. There are a couple of website resources that list multiple vendors with profiles of products, but they tend to be largely derived from the vendors themselves, not any third party review, and out of date in many cases.

	Considering scheduling, making effective use of the most precious non-renewal resource there is-time-is critical. Quick Internet reviews of corporate websites and industry articles was employed to pare the candidate list down to 25 or so. Scheduling demos and reviewing that many would take months, so further filtering was conducted using simple criteria:

 More than a few have become defunct or been acquired over recent months and years.

Are they in danger of going out of business? 

Have they been delisted from the stock exchange? Have they sustained losses? Have they been shedding staff? Are they on the market for sale?

Many don't offer this (or don't have a good one) but would love to try and sell you a full-service offering.

Are they a CRO that happens to have and EDC system or a technology company?

 Sometimes it makes sense to use a CRO with and EDC system, but for most DIY solutions it's probably best to stick with pure technology companies (they have an incentive to make their product good so you don't call them-whereas hourly billing companies love phone calls).

What do your colleagues think? What have you heard about town? What does your CRO (if you have one) think? 

Utilize your network to seek out colleagues who have used different systems and get their unbiased opinions on ease of doing business with the vendor, ease of use of the product, problems, issues, etc. Your CRO can be invaluable at this point too. This criteria should be used to exclude vendors from consideration, not select them-your needs are different than anyone else's.

Is their base technology from this century or the last? 

This can be hard to detect without specialized knowledge.

	Using the simple criteria above and doing a bit of digging on the Internet and some time on the phone, a list of seven was formed. One, on subsequent examination, didn't truly have a DIY option and tried to obtain customers with vague generalities and then present their full-service product as a viable alternative. Six companies remained.

	To deal with disparity between systems and pricing variation, standardization of data was required. Not to standardize, given the variation in system capabilities, look, feel, and even sales representative expertise, leaves you vulnerable to primacy and recency effects (the first seen is best or the most recent-usually the latter, as it is very hard to organize non-standardized data and make valid comparisons, and thus the most recent thing is best remembered).

	To ensure standardization of evaluative data an 87 point assessment tool was developed that highlighted capabilities in the arenas important to the company's particular research needs, as well as fundamental system characteristics. This approach meant that "canned" presentations/demonstrations did not meet my needs, which caused some sales representatives some difficulty. In addition two "model" projects were presented to potential vendors and pricing for those and only those was sought.

	In the first case, the assessment tool covered the areas listed in Table 1. The assessment grid was made available to vendors prior to the demo taking place, and they were asked to fill out responses insofar as they were able. When the responses were received back, if the answer was clear it was scored as follows:

		0 = feature or characteristic not present

		1 = feature or characteristic present but difficult to use, poorly implemented, limited functionality etc.

		2 = feature or characteristic present in good usable form/implementation

		3 = feature or characteristic present in such a novel form that it enhances productivity or system desirability dramatically

	The last score of "3" was only awarded twice out of over 600 total responses, and where another product had the same utility or outstanding character both scored "2." The assessment tool then guided the subsequent demonstration and any remaining questions/unclear responses scored. It was only after the assessment tool had been completed that any other features the sales representative wanted to highlight were reviewed for information only. It is important to note "that feature is coming" or "it is in our pipeline" counted as zero-updates to complex software systems often take far longer than expected and in many cases prove impossible once attempts to program begin.

	Apples to apples pricing was obtained through use of strict models as seen in Table 2. Some characteristics of the models such as unique CRFs and data fields were kept constant deliberately in order to determine if there were any economies of scale that became evident at higher volumes of data and greater duration. All the data fields were assumed to have at least one edit check or conditional action associated. Importantly, these models were treated as "standalone" projects, that is, any license fees or training that would not necessarily be charged for multiple trials was added in. Overall, this methodology allowed for direct comparison of pricing across vendors.

	Dealing with the technical side of specialist knowledge and sales reps-essentially making yourself an informed consumer-is more problematic. If you are a highly experienced EDC user with knowledge of multiple different systems, then no outside assistance may need to be sought. Often, in a small business environment used to running paper trials, that expertise is simply not available. In this case, soliciting help and feedback from your CRO may be invaluable. It may be wise to consider retaining an independent consultant-but be warned, many "consultants" are actually resellers or quasi-distributors of a single EDC system and will be inherently biased; reseller consultants may be of some value in implementing the chosen system and assisting managing process re-engineering. A few key points to look out for are described later in this article, but do not by any means form a comprehensive listing.

	Scores, across the board, were fairly even, surprisingly. There was one outlier with a markedly lower score than the rest of the population. Aggregate grid scores by vendor can be seen in Figure 1. Data in tabular form against maximum possibles by area and total can be seen in Table 3.

	It is clear from the data in Table 3 that despite Vendor F being superior overall, they lead only one category (data/build features) and lag the pack in some others. In order to determine the meaningfulness of the scoring data, scoring characteristics were plotted to determine the standard deviation (Excel 2003 STDEVP function) which yielded the results shown in Table 4. Vendor C was removed from calculations-see discussion.

	On the assumption that a score that deviated from the median value more than one standard deviation was meaningful (or at least "noteworthy" given the sample size), the "above" and "below" par performances were calculated by vendor.

	Pricing results were interesting, to say the least. As can be seen in Figure 2, a wide variation exists between vendors-pricing was not obtained from Vendor C as they were removed from consideration early (see discussion).

	Vendors, in general, were reluctant to provide pricing, and in some cases it took a number of communications to source this important component. Model 1 results, for the five site, 50 patient 12-month single-arm trial were somewhat expected, with the exception of Vendor F whose very low price prompted a phone call in which it was confirmed as accurate. Vendor A did not provide pricing for Model 1 despite being asked to do so on a number of occasions.

	Model 2 results were most surprising and unexpected. While one expects to see some cost increases for a longer, larger, and two-arm trial (50 sites, 250 patients, 36 months, two arm), the 18x multiple for Vendor F was very surprising (confirmed by e-mail as accurate). Vendor B gave a multiple of 4.4x, while Vendors D and E were just under 3x. The objective of the two models was not only to standardize for comparability, but also to give an indication of economic scalability for a DIY option. In this case, clearly Vendor F is poorly scalable, with Vendors B, D, and E progressively more so.

	It is important to note in these results, however, that Model 2 pricing revealed even more important information than economic scalability-in fact, the pricing given called into question the entire premise that DIY is a cheaper option than full-service EDC. Other full-service research conducted but not presented here indicates that a full custom-built-for-the-project EDC system can be had for Model 2 starting at around $160,000 to about $220,000. Given that in the DIY model it is incumbent upon the sponsor to provide all the project management, CRF/study building, testing, and site training (that is, a personnel overhead that needs to be financed), the prospective purchaser must seriously consider exactly how much research they need to conduct and of what type, assuming Vendors B and D products will be an economic solution for moderate-larger trials is simply not true.

	In this particular instance, Vendor E is superior numerically and is highly competitive in their pricing, with moderate scalability. Also Vendor C is markedly below par across the board-this vendor also failed a basic security test: after logging out of the website, clicking the "back" button on the browser allowed not only viewing of the last CRF used, but also permitted manipulation and saving of that data and further use of the system, without reentering user name or password. It would seem then that the choice is clear, but a flaw in methodology became apparent during analysis. Vendor C was excluded from all further analyses once this basic failure was discovered.

	The methodologic flaw detected was one of weighting, or the absence of it. When Vendor C was removed from the dataset variability across vendors is very low. Scores remaining are: 90, 94, 81, 89 and 96 (Vendors A,B,D,E, and F) for a standard deviation of 5.79. This excludes Vendor D as probably inferior and leave the remainder essentially even in overall scores. How then to decide amongst the remaining candidates?

	In the assessment process the primary effort was in standardizing the data by executing as objective a process as possible. Weighting is, by its very nature, subjective. It could be argued that a form of weighting is present in that the areas of interest do not have an equal number of questions. Additionally, the actual questions chosen represent some form of subjective selection in the first instance. Nonetheless, it emerged that the questions themselves had different inherent values that would-and indeed should-influence the results; some questions were simply more important than others and should be weighted as such.

	For example, one vendor who scored well cannot support any Internet browser other than Microsoft Internet Explorer (IE)-this effectively would exclude users of Apple based systems (Safari browser bundled with OS), or those precluded by their IT departments from using IE for security or other reasons from participating in any trial developed by the system. Another vendor had restrictions on what browsers other than IE and FireFox could do-Chrome, Safari, and Opera had limited functionality. These restrictions could effectively hamstring any clinical trial conducted where IE or FireFox were not used-which may be a lot of sites, especially in academic ones. Browser agnosticism is very important, as there is increasing diversity of browser usage, and to remain focused on one or two products may exclude large segments of potential investigative sites. Thus, client software agnosticism, in this case, browser type, should probably carry more weight in ranking. Figure 3 represents current worldwide browser usage.

	Assessment questions were posed with respect to known issues-in this case, use of Java. Unfortunately, Java applets carry major security risks, and many institutions completely disallow Java use. Another general area of concern is ease-of-use for the sites-investigators are well known to strongly desire multiple form sign-off with one electronic signature, and to not have this feature threatens their willingness to participate in the trial.

	Some systems share a common software code base-that is, anyone using the system shares fundamental application code with others. This is not a bad thing in and of itself, but when the vendor changes the code base whether you like it or not-a forced upgrade schedule in effect-changes can be introduced that require instant SOP revisions, new site training, and/or new staff training, and may have a severe impact on productivity. In the vendor list examined in detail above, two employed a shared code base with forced upgrades (B and D), one had a partial shared code base and was prepared to negotiate code change release schedules (F), and the rest provided a unique instance of the code to the sponsor who could determine if they wished to upgrade or not. Much of the time, a forced upgrade schedule is not problematic, but it can, and does, have ramifications at times that you'd probably rather avoid, so weighting of this component may be appropriate.

	Other vendors will not allow "sandbox" use of their software-that is, the ability to use a fully featured system, without obligation, in order to evaluate usability with staff and others at your own pace without vendor representatives present. This peculiar notion-after all, you're going to spend many thousands of dollars and even a car dealer will allow you to test drive a vehicle that costs far less-limits your ability to assess the system to what the sales representative is willing to show you in a highly-sanitized "demo" environment. For a DIY system, I would class the availability of a "sandbox" as extremely important-not, perhaps, to the "must have" level, but close.

All of the above points formed part of the assessment tool, and many more besides. Capabilities vary wildly across systems and it is almost impossible to find one that does everything you'd like. Very careful consideration must therefore be given to what is an absolute requirement, a very important requirement, desirable requirement, an undesirable feature and a must-not-have feature/characteristic, with the responses weighted accordingly. Also consider carefully, what weight does price carry? Is it ok to pay twice as much for a system with one more important requirement? How about two? Three? Given that very few systems will meet all your requirements, after weighting, a conjoint analysis may have to be performed in order to assess which system truly is "best" for your particular needs and circumstances.

	While much of the above is applicable no matter what environment you find yourself in, there are a few more considerations for the smaller enterprise to contemplate. For example, the CDMS had inventory and site payment capabilities. In this small company environment, selection of a system without those features means personnel have to manually perform the function, or consideration be given to spending extra to have data from the EDC system populate the CDMS for processing. That extra burden in people hours or money may present problems that are much more easily absorbed by a large organization. Additionally, if the skill-set required to build studies in a particular product is highly specialized with a steep learning curve then the impact of one staff leaving in a small department will have a far greater impact than in a larger department-thus no matter how powerful the product, it may make you inherently vulnerable to staff turnover.

	Finding and assessing an electronic data capture methodology is a task that must be approached methodically in a structured manner. EDC systems vary so much in capabilities that the only valid way to make good judgments on suitability is via standardization of the method of assessment. This is especially true for the smaller life sciences company which may not have the in-house expertise to rely upon, and seeking CRO or consultant help may be invaluable. Enabling cross-vendor pricing comparisons through use of models can reveal critically important information on product scalability and economic feasibility for different project types. Finally, careful consideration of your company's specific needs may lead to weighting of particular components or dimensions of DIY systems which can help differentiate vendors when a clear "winner" is not present.

, PhD, is a Healthcare & LifeSciences Solution Design Consultant at PowerObjects, e-mail: 

Feature Article

Options have emerged that make DIY EDC technology more accessible to smaller organizations.

DIY EDC Evaluation

The sponsor of a recent Phase II global study, conducted at more than 50 sites, invested in a centralized digital pathology core lab to ensure a homogenous and representative study population. For this study, the lab, powered by Biomedical Systems, worked to confirm the pathological diagnoses of patients' disease states.  

The investment proved advantageous, and the findings were startling: 30% of patients submitted for central review did not meet the inclusion criteria. Failure to properly classify its patient population can leave sponsors with a significantly underpowered study—rendering the drug manufacturer with inaccurate efficacy results, a failed study, and a heavy price tag. 

To understand how the study sites, which relied on patient medical histories combined with site-prepared histology biopsy slides, approved patients who did not meet the inclusion criteria requires an understanding of the digital pathology evolution. 

Before the advent of slide digitization, pathologists reviewed specimens on glass slides under microscopes. To secure multiple pathologists' interpretations, labs would prepare several sets of slides from the patient's tissue block—each slide containing a different level of tissue shipped to the reviewer. Variance in tissue levels triggered variability among inter-reader interpretations of the specimens. Subsequently, sponsors began leveraging digital pathology to provide reviewers with identical digital images of patients' tissues, effectively reducing variability. 

In the referenced Phase II study, variability could explain the 30% discrepancy between internal site review and the central review process. With the study, pathology slides were reviewed at more than 50 sites; each of the sites employs its own pathologists. In addition, each site offers its own equipment and methods for slide collection and patient history documentation. 

With centralized digital pathology, the study and its readers must follow rigorous, standardized processes focused solely on inclusion criteria. Pathologists may only assess the morphology depicted in the digital slide image sets, which reduce variability and bias.  

For the Phase II global study, readers were trained on standardized equipment, scoring methodology, and the electronic case report form application. Each reader in the study viewed the exact same digital slide image sets. Readers followed proven project management practices and systems to ensure their assessment results were directly linked to the evaluation. The electronic case report form guided each of the readers through the same protocol-specific evaluation and grading scale.  

After a reader completed each required field, he or she would enter a given password to apply a digital signature and date/time stamp to the completed record. The completed record was sequestered, preventing the reader from returning to the record or the digital slide image set. These processes effectively reduced variability, promoted homogenous analysis methodology across all readers, and ensured complete records were provided in a format conducive to statistical analysis. 

			 By centralizing a study, sponsors and its study partners can typically decrease the number of pathologists needed to evaluate patients for inclusion. This reduction in pathologists lessens the variability associated with the inclusion assessments. In the study, the centralized lab effectively worked to significantly decrease the number of pathologists evaluating patients. Reviewer reduction was coupled with standardization—providing uniform training, equipment, and review processes to improve patient inclusion percentages and reduce variability. 

 In the study, the independent review, which is inherently part of the centralized review process, proved effective in ensuring only patients who met the protocol inclusion criteria were enrolled in the study.  

In cases of site-selected inclusion, site pathologists review patients' history, test results, and histology primarily for patient care. Study inclusion criteria often serve as the site's secondary goal. The pathologists' expanded patient knowledge, which is important for patient care, can potentially hinder the clinical trial process by injecting unintended bias into the patient inclusion evaluation.  

The readers were required to follow  a centralized digital pathology process, and only focused on the morphology represented within the digital slide images. For the study, readers were blind to all extraneous patient information.  

The impact of an underpowered study can cost drug manufacturers millions of dollars and their reputations. Centralized digital pathology, in this example, reduced bias and variability while increasing the probability of accurately assessing efficacy and statistically relevant data. 

Tim Callahan, PhD, Chief Scientific Officer, Biomedical Systems. Member of Applied Clinical Trials' Editorial Review Board.

The sponsor of a recent Phase II global study, conducted at more than 50 sites, invested in a centralized digital pathology core lab to ensure a homogenous and representative study population.

Central Lab Review Prevent Underpowered Studies

Last September, this column looked at the prospects for support to clinical trials, in Europe and in Africa. 

Things don't move very fast in the European Union, and many of those prospects remain just that—prospects. But at least one element slotted into place at the end of January to give shape to a promise to help clinical trials in Africa. Researchers from the continent's sub-Saharan countries are going to have better chances of upgrading their clinical research skills after the European drug industry signed a memorandum of understanding with the European & Developing Countries Clinical Trials Partnership (EDCTP). 

What the deal will offer, as from later this year, is a series of research fellowships in pharmaceutical companies in Europe for clinical and non-clinical staff from the region. Each company will design a customized program that reflects its own activities and expertise, and the field of interest of each trainee, and will include short- and long-term goals as well as clearly defined ethics and compliance rules.  

The fellowships will be awarded through open calls for proposals. Candidates will be selected by EDCTP, and those who are selected will be able to spend up to two years in a company, very largely a company of their choice. 

The memorandum spells out the rationale for the initiative. It says that the research-based pharmaceutical industry "has developed skills related to clinical trials that cannot be acquired through academic training or medical/scientific practice."  It makes the point that within European-based pharmaceutical companies, crucial activities "are performed to a very high standard," citing study design, biostatistics, data management, project management, study monitoring, report writing, regulatory submissions, and audits. The intention, therefore, is to offer sub-Saharan African scientists who are actively involved in clinical research projects the opportunity to gain hands-on experience in a pharmaceutical research setting. 

The full details of the scheme will be released in February, but the outline conditions are that candidates should have a guaranteed position at their home institution, adequate skills, and experience—which will be assessed on a case-by-case basis—and will be expected to return to their home institution for a minimum of two years after completion of their training. EDCTP will cover travel and subsistence costs. The companies that take part in the scheme will donate time spent by tutors, materials, and access to internal training opportunities, and up to €5,000 for a fellow's business travel, such as to attend conferences.  

Richard Bergström, the Director General of the European Federation of Pharmaceutical Industries and Associations, said at the signing ceremony that the arrangement was an opportunity for companies as well as for researchers from Africa. "EFPIA's members will benefit from interaction with African scientists from diverse backgrounds beneficial to the development of new or improved treatments in poverty-related diseases."  At the same time, "We hope to help them acquire skills relevant to achieving their research and professional goals."  

Professor Charles Mgone, EDCTP Executive Director, said it was "critical that more African researchers are sufficiently trained and equipped with skills and expertise in clinical trials research in the fight against diseases that severely impact the social and economic development of sub-Saharan Africa." He declared himself "delighted" with the deal.  

He might well do so. Because it is at least one concrete result from so much talk about the merits of, and need for, support to medicines research in and for developing countries.  Some of the other initiatives are taking a long time to bear fruit. As Mgone himself commented at the signing ceremony, the deal with EFPIA companies will reinforce collaboration in the recently agreed EDCTP program—the second phase of this 10-year old initiative. That is, as and when the second phase can really start. Because this was one of the promising prospects that brightened the future of EDCTP during 2012—as 

 commented in September. Unfortunately, the broader context for turning that promise into reality (which is, in crude terms, largely a question of money) remains inchoate.  The overall EU spending plan for the next seven years should have been agreed last year, and any meaningful action in the EDCTP's second phase depends on precisely that agreement for most of its funding. But dissension about how much to put into the general purse persists among austerity-hit EU member countries—over issues more related to payments for farmers or allocations for regional development, rather than the relatively obscure area of clinical trials in poorer countries. And this is delaying the necessary strategic consensus on how much the European Union will spend on anything that happens between the end of this year and the end of the decade. 

The new fellowship scheme has been created in cooperation with the European Commission, which has repeatedly declared its interest in promoting international standards for conducting clinical trials, and in boosting research capacity in Africa. And at present, that is about all the European Commission is able to do until it is given a clearer view of future spending plans by the leaders of the national governments that ultimately call the shots in the European Union. So for all the noble aspirations to see the second EDCTP program make new progress in the development of treatments, vaccines, and diagnostics for HIV/AIDS, tuberculosis, malaria, and neglected infectious diseases, right now the best bet is to get a few researchers a few months in the clinical development divisions of a few drug firms in Europe.  

The EDCTP initiative is not the only victim of the failure to agree the European Union's future funding plans. A series of plans to give financial support to clinical trials in Europe, agreed in principle as a good thing a year or more ago, is also on hold, because these plans too are held hostage by the paralysis over strategic spending. The EU ambition to put some $100 billion behind a new European research program—in which health is one of the priority areas identified, with some very specific provision for clinical trials—is stuck in the same logjam as the EDCTP second program.  

So all the fine talk last year about  funding for "the development of better methodologies to allow trials to focus on relevant population groups, including those suffering from other concomitant diseases and/or already undergoing treatment," have so far come to naught. The same is true for plans for "the determination of comparative effectiveness of interventions and solutions," for "enhancing the use of databases and electronic health records as data sources for trials and knowledge transfer," for "improvement of cross-cutting support technologies for drugs, vaccines, and other therapeutic approaches, including transplantation, gene, and cell therapy."  

Meanwhile, things are still moving only slowly on the European Union's bid to update its clinical trials rules. As already (indeed repeatedly) remarked in this column, the European Commission's proposal of last July has yet to command universal support. Ireland, the incoming presidency in the European Union's rotating chair for the first half of this year, pledged in its program to "make progress on the proposal for a regulation concerning clinical trials on medicinal products for human use."  It remarked that "this proposal is designed to address the fall in clinical trial applications in Europe, which are essential to providing new innovative medicines for patients and maintain a strong pharmaceutical industry, by reducing the administrative burden and costs on sponsors, while at the same time ensuring that subjects are fully protected." But it has found that the discussions it chairs in the EU Council are tough work.  

Many member states feel that the commission proposal is just too glib, requiring, as it does, that they should do all the hard work at national level—in reaching a single opinion on every application for a clinical authorization—while the European Union will then take the credit for a smoother system to which it will, in effect, make only a minimal contribution on each application. The risk is that the entire legislative process may be held up by demands for a fairer share-out of the workload, with national authorities seeking that the commission should, for instance, take a more overt role in defining procedures for authorization, and the commission sticking to its view that member states should sort out their own national procedures.  

The problem now springs from the fact that EU legislation has to fit the rhythm of broader EU governance processes, and fresh elections to the European Parliament are due in mid-2014. For new rules to go through, the parliament will have to give its approval to whatever deal might emerge to resolve these divergences, and that could take months. Given that parliament's interest in current business falls off as the pressure for electoral campaigning rises, a real race to the finish is in prospect, even if member states and the commission find a compromise. And if the proposal does not get passed by parliament before parliament dissolves for elections, it is likely to be declared dead—and the whole process will have to start again in late 2014 or beyond. 

By their nature, clinical trials are geared to demonstrating future benefits. Any benefits from EU moves in support of clinical trials seem, at present, to be well into the future—so far in the future as to be very hard to see.

 is a freelance journalist who specializes in European health affairs and is based in Brussels, Belgium. 

View from Brussels

Researchers from sub-Saharan countries are going to have better chances of upgrading their skills.

Bringing Brussels and Africa Closer on Trials

Drug development costs are higher and timelines longer than they have ever been. Study protocols are complicated by factors including the complexity of international markets, which have unique regulatory requirements and economic conditions. Still, researchers must assure that clinical trial planning and execution proceed effectively and efficiently. 

Organizations can better decide whether to launch a product or terminate its development by bringing together their best data and best minds with best practices. 

Many pharmaceutical companies apply three basic principles for improved clinical trial-based decisions. These decisions range from clinical development plans for a given indication—making sure the right questions are asked and answered—to which indication to investigate first, to which CRO should carry out the study. Three decision principles can reduce risk and increase the probability of drug development success: 

When scientists, physicians, quality assurance, compliance, marketing, and operations stakeholders make decisions together, their objectives compete. Some are based on structural incentives of their role, others by individual preferences/biases. All are shaped by good, but often incomplete, information. Decision makers must collaborate to structure their decisions. With a framework for choices, stakeholders can collectively weigh in on the relative importance of factors, gain insight into the positions of others, and highlight and diffuse biases to reduce knowledge gaps.  

Using techniques like paired comparisons of criteria, stakeholders can draw out points of agreement and disagreement and better learn what drives their choices. This fosters fair process, consensus building, and common language, and disarms advocacy-based decisions by framing issues important to the choice. Performance of options can be clearly tied to key factors, and objectives and priorities can be linked to courses of action. 

Decisions rarely remain static for long. Considerations change as new evidence is collected. Financial considerations before and after an economic crisis, or regulatory considerations on either side of an election or court ruling, can be very different.  

The performance of options against key criteria can be independent of the importance of criteria. With the collaborative prioritization described above, relative importance of criteria can be adjusted to reflect actual or hypothetical future scenarios. Testing decisions this way, stakeholders can understand how fragile or robust their choices are and ask questions.  

 A repository of prioritized factors from previous decisions allows you to see how a decision was taken. A decision map can compare the future outcome of the decision to the usefulness of the framework used to make the choice.  

Mapping lets decision makers assess the accuracy of their predictions, and how well factors were understood. Mapping also helps improve decisions over time. Stakeholders can observe whether perceived success factors were truly influential, better understanding the factors that affected the decision—and alternate courses of action if those factors had been anticipated. They can also begin to understand the value of their estimates or forecast of performance against key decision criteria. 

By applying these practices in a structured decision making framework, pharmaceutical decision makers can increase collaboration, transparency, efficiency, and effectiveness in clinical research decisions, while reducing development risks. 

 Executive Vice President of the Decision Solutions Group Decision Lens E-mail: 

A Closing Thought

Decision makers can increase collaboration, transparency, efficiency, and effectiveness, while reducing development risks.

Three Steps to Better Decisions

The 10-year increase in the number of unique and total procedures per protocol corresponds to an increase in the number of endpoints supported by these procedures. Research conducted by the Tufts Center for the Study of Drug Development (Tufts CSDD) finds that between 2004 and 2012, the number of endpoints per protocol has nearly doubled from an average of eight per protocol in the earlier period to 14 endpoints in the later period. Half of all procedures support supplemental secondary, tertiary, and exploratory endpoints—a proportion that has remained constant during this eight-year period. Sponsor companies have added more procedures for a variety of reasons: clinical scientists seek more contextual data to interpret and validate findings and guide development decisions; statisticians wish to collect more data to explain observations and potential new interactions; and sponsor companies also note that procedures supporting tertiary and exploratory endpoints are added in anticipation of perceived or real regulatory agency and payor inquiry, commercialization, and competitive pressures. 

—Tufts Center for the Study of Drug Development, 

Between 2004 and 2012, the Number of Endpoints per Protocol has Nearly Doubled

The Food and Drug Administration approved nearly 40 new drugs in 2012, beating most recent totals, and is looking for provisions of the FDA Safety & Innovation Act (FDASIA) to further improve its operations and encourage innovation. Prescription drug user fees (PDUFA) authorized by FDASIA provide the agency with more resources to assist sponsors in proposing appropriate research programs and in filing complete applications, all designed to revitalize the development of critical therapies. 

FDA had already approved more than 30 new molecular entities (NMEs) by early December 2012, putting it ahead of a healthy performance in 2011, reported John Jenkins, Director of the Office of New Drugs in the Center for Drug Evaluation and Research (CDER), at Elsevier's FDA/CMS Summit in December 2012. Jenkins predicted more approvals before year-end, and he was right: the agency approved three more drugs the next week and five more by December 31, bringing the annual total up to 39 new drugs and biologics. Many of the new treatments are directed as life-threatening conditions such as a rare leukemia, inhalational anthrax, Cushing's disease, and drug-resistant tuberculosis.  

There's no more drug lag with Europe, moreover, Jenkins added, noting that more than 80% of last year's NMEs were first approved in the United States. And a continuing rise in commercial INDs filed with FDA promises more new drugs in the future. 

The rise in approval numbers reflects an efficient review process, as Jenkins and his staff strive to achieve more first-cycle approvals. FDA approved more than 70% of priority applications in one cycle in 2012, and 60% of standard NMEs, a task made easier by applications for targeted therapies with large effect size and for rare disease treatments with clear benefit/risk ratios. In addition, sponsors are filing more complete applications in the first place, Jenkins commented. The days are over when CDER will accept data from a second pivotal trial six months into the review process.  

The development of more critical therapies for serious diseases may be encouraged further by guidance in the works that aims to clarify the opportunities and requirements created by FDA's various expedited approval pathways—fast track, breakthrough, priority review, and accelerated approval. And new liaison staffers will offer assistance to firms navigating the FDA review system, which should be particularly helpful for the growing number of "emerging sponsors" filing their first new drug applications.  

Bob Temple, CDER Deputy Director for Clinical Science, noted that FDASIA provisions supporting approval of breakthrough therapies based on limited clinical evidence are not really new because FDA frequently has accepted more risk and shown flexibility with highly innovative products. But the new policy is helpful because it clearly instructs CDER to facilitate approval of "drugs with impact." The rise in new drug approvals, he noted at the Elsevier conference, reflects an increase in very well-targeted cancer drugs with "huge effects" in specific populations, an approach that he hopes can be applied to other conditions, such as asthma. 

Temple anticipates that more targeted therapies will come to market through wider use of enrichment strategies in clinical trials, as described in new guidance issued in December 2012. This draft document describes methods for decreasing heterogeneity and for selecting patients likely to have a certain condition or to respond to treatment, strategies that Temple has supported for years. The guidance advises sponsors to acknowledge their use of enrichment studies in protocols, to avoid labeling that overstates benefit for other populations, and to be prepared to provide data on response in marker-negative patients. Enriched study methods offer opportunities to bring critical therapies to patients based on smaller, faster clinical trials, says Temple: "While enrichment won't save a drug that doesn't work," he notes in an FDA blog, "it will help find one that will." 

One concern at FDA is that approval of new breakthrough drugs could be delayed by problems with manufacturing scale-up. CDER director Janet Woodcock described FDASIA's breakthrough therapy program as "a game changer" for cancer, genetic diseases, and other serious conditions at a November 2012 conference sponsored by Friends of Cancer Research and the Brookings Institution. But even if FDA wants to approve a highly targeted cancer therapy that demonstrates complete response in Phase I or II, it may have to wait until the product meets "ready for manufacture" criteria. Sponsors anticipating such problems should meet early with FDA specifically to address manufacturing and scale-up challenges. 

The Food and Drug Administration approved nearly 40 new drugs in 2012, beating most recent totals, and is looking for provisions of the FDA Safety & Innovation Act (FDASIA) to further improve its operations and encourage innovation.

FDA Speeds More Breakthrough Drugs to Market

, (Raleigh, NC), a therapeutically focused global clinical research organization announced the appointment of 

as its new Chief Information Officer. Chiovari will lead the company's worldwide information technology initiatives, including the development of new information systems architectures and strategic plans aimed at improving efficiency for clinical development programs. 

CTI Clinical Trial and Consulting Services 

(Cincinnati, OH) announced the hiring of 

 as Study Manager. CTI also announced the promotion of 

(Research Triangle Park, NC) announced an expanded business development team with the addition of three Directors of Client Service: 

 as Director of Finance/Chief Financial Officer. DIA also announced that 

, former CEO of the Danish Medicines Agency, has been appointed Director for DIA Europe. 

 (Souderton, PA) Clinical Technologies business unit announced that 

has joined the company as Vice President of New Products and Services. Morris will oversee the development of Almac's IXRS technology and enhance the company's metrics-based discipline to the software development process. 

 (Morrisville, NC) announced company executive 

, RPh, Vice President, International Business Development, will take on additional responsibility as the Managing Director, EMEA. 

, (Princeton, NJ) inVentiv Health's clinical segment, announced that 

, PhD, has joined the company as Executive Vice President of Clinical Operations for Phase II-IV Development in Europe, Middle East, and Africa (EMEA), Asia-Pacific, and Latin America. 

 (Costa Mesa, CA) announced the appointment of 

 to the newly created position of Vice President of Clinical Operations. 

Applied Clinical Trials-02-01-2013