INTRODUCTION

Before you start counting the money from your ‘imminent’ pharmaceutical deal, you may want to take a look at your bits and bytes first. Pharma companies face increasing pressure from investors when it comes to the approvability of new biotechnology products and the US Food & Drug Administration (FDA). The best deals will increasingly go to those biotechnology companies who have reduced pharma and investor-perceived risk when it comes to new molecular entities. And one area increasingly under the risk microscope is the biotechnology firm's record integrity.

RISK-AVERSE INVESTORS

A June 2008 report noted that 78 per cent of pharmaceutical executives in both the US and the EU increasingly worried about the growing caution of the FDA in granting approval for new biotechnology drugs.1 Although there is little doubt that biotechnology medicines remain the linchpin of life science industry growth, given the intensifying competition from biosimilars and the less than stellar record of biotechnology pipelines panning out in late stage clinical trials, key investors are growing dubious of pharmaceutical companies that simply rely upon buying-out or licensing new molecular entities without some level of proven track record. Many of the pharma executives I have spoken with, some of whom have been burned by poor endgame clinical results from their newly licensed biotechnology drug candidates, are taking a page from the regulatory playbook of Shire PLC and examining the integrity of the prospective biotechnology partner's laboratory and clinical work; a caution that is also being driven by the FDA's recent emphasis on record integrity and research data quality in preclinical and early stage clinical trials.2

Typical new drug or biologic development follows a multi-year path starting with fundamental research and development work to simply identify potential molecules that may help fight against a given disease or condition. Next comes the ‘preclinical’ phase when the company really examines the molecule in light of its eventual use in human beings. Preclinical work can be a broad mix of in vitro, in serum and animal testing, all of which lays the foundation for the first in man studies known as Phase 1 clinical (or early stage clinical). For such early clinical tests, typically no more than 90 people are involved, whereas the later two phases of clinical trials (Phases 2 and 3) see the drug or biologic progress to hundreds and then thousands of patients. For many new biologics targeting narrow niche diseases with small potential customer bases, a large-scale Phase 3 trial may be difficult given the small total patient population; for these narrow niche biotechnology firms, especially the data integrity of their preclinical and Phase 1 trials is paramount.

DATA INTEGRITY AND THE FDA

Over the past several years, FDA officials have given several presentations on their concerns when it comes to research data quality. As early as 2003, Stan Woollen, the Associate Director for Bioresearch Monitoring at the FDA, discussed agency concerns over preclinical and early stage clinical data quality, especially in light of its use as the foundation for the rest of the commercialisation activities, decisions and data that go into the eventual submission to the agency.3

When it comes to research record integrity, the FDA (and thus pharma) is concerned about three key issues: uninformed ignorance, random sloppiness and deliberate fraud.

Uninformed ignorance stems from a genuine lack of awareness that a particular action is wrong or inappropriate. Examples include reporting transcribed data (rather than the original test data) as the original raw data, signing or initialling a laboratory notebook4 page long after page was originally written, backdating a form missing its date, and so on.

Random sloppiness introduces unintentional errors that influence test results. These mistakes most commonly result from inattention to detail, lack of supervision, rounding of numbers, data estimations rather than actual measurements and transcription errors. Because it is often difficult to determine if data errors are the result of accidental oversight or intentional fraud, sloppiness in particular tends to weaken the analyses and conclusions drawn from such impacted tests.

Most worrisome to FDA officials, however, continue to be the real-world examples of fraudulent data submitted by individuals. In the past few years, examples that investigators have uncovered include creating data that were never actually obtained, altering test results to be more favourable and omitting data that were unfavourable. For instance, if six assays were conducted, two resulting in unexpected data and four resulting in data within the expected parameters under the original design of the tests, only reporting those four ‘good’ results is deemed by the FDA as fraudulently representing test results. Recent court cases have also called into question the record integrity of biotechnology–pharma partnerships, particularly when it comes to the reporting of results.5

FDA inspectors have been trained to review at least 12–18 random pages in any lab notebook, plus test protocols and any electronically captured raw lab data. Expect at least this much scrutiny from any pharma due diligence team. It is not unusual for a prospective pharma partner to try to replicate some of the biotechnology firm's testing, even going deeper into more variability. Pharma's assessment of the biotechnology firm's testing competence and data quality will influence the value of any deal.

To be able to negotiate from a position of strength with any prospective pharma partner, biotechnology executives must be able to demonstrate the integrity of records that support their intellectual property.

RECORD HANDLING PROCEDURE

The best way to assess record integrity and close any gaps is to examine core data results that support the new biologic's safety, quality and effectiveness, and ask, ‘What are the risks of our data from the pharma's standpoint and how do we prove our record integrity?’ Consider doing this in a chart fashion that might look like Figure 1.

Figure 1
figure 1

Sample record integrity chart.

Full size image

This is a simplified version of a more complex record integrity charting exercise that I work through with my clients. The more complex form of this addresses metadata linkages, data storage failures, records management and handling policies and procedures, equipment qualification and calibration proof, raw material quality assays, and so on.

In Figure 1, Your records include the test results, analyses, raw data, protocols, and so on. Risks are the problems that a prospective pharma partner will look for when assessing biotechnology competence and data quality. And proof gives a prospective partner-tangible evidence of data integrity, provides confidence in the biotechnology firm and improves their chances of negotiating a favourable deal.

For most situations, the three most critical areas to review in preparation for pharma negotiations are record accuracy, completeness and traceability.

RECORD TRACEABILITY

The traceability of information speaks to four questions: who, when, where, and how. Who did the work? When was it done? Under what conditions and how? Records are proof that answer these questions with clarity. Therefore, for any preclinical and early stage clinical records, a reviewer must be able to quickly ascertain the clarity of this information … and identify potential red flags.

When auditing records, some of the traceability issues I look for are as follows:

  • missing dates;

  • missing times; and

  • unsigned (or initialled) laboratory notebook pages or forms.

Errors or weakly support conclusions that are quickly blamed on a now gone post-doc, summer intern or lab technician raise my level of scepticism. The blame may well be valid, but should not the sloppiness have been uncovered earlier by the post-doc's supervisor? A biotechnology firm with poor internal oversight of its own research has weakened its own hand for any pharma negotiations.

One additional verification may be in order and the simplest, least expensive method is for the biotechnology executives to assess whether testing protocols conformed to recognised standards or regulations (such as those in the FDA's non-clinical Good Laboratory Practices). Fortunate indeed are those biotechnology companies who can clarify this and document it. Conformance of methodologies to internationally recognised standards, particularly when it comes to traceability, goes a long way towards establishing record integrity and encouraging any due diligence auditor to look elsewhere.

DATA COMPLETENESS

The second area to examine before initiating discussions with a prospective pharma partner is record completeness. Typically, I will select half a dozen or so summary reports and then follow their trails – summaries, conclusions, analyses, raw data, protocols (and protocol changes or amendments), laboratory notebooks, equipment and materials listings, and training records, just to name a few.

I am not looking at the scientific validity of each of these items – so much of this work is far too specialised for any auditor to assess its scientific validity (that is one of the reasons the FDA relies on outside scientific panels). Rather, I am looking for completeness.

For instance, when an item (such as an image from an electron microscope or a series of time-lapse photographs) is listed, I ask for the actual item itself – the photograph from the microscope, the X-ray, the raw chromatography data. I still remember the look on the face of a laboratory director the day he discovered that various images referred to in a crucial laboratory report were missing. Three days of frantic searching did not find them. As tensions rose, a full-company meeting was called and the missing photographs discussed. Following the meeting, the network administrator quietly confided in his manager that the photos had been archived off the network months beforehand as the photos were taking up too much space; he had then forgotten to inform anyone (other than jotting it down in his notebook). This is an experience that any biotechnology executive would do well to avoid the day before any pharma due diligence team arrives.

Once I have the raw data files in hand, I examine them looking for clues of gaps or omissions. Have all the results been captured or noted on any graphs? Are the results within expected tolerances? Are there missing results unaccounted for in the summaries? Have any of the numbers or results been averaged together? Every outlier (or groupings thereof) should have some sort of reasonable explanation attached to it in the research analyses or supporting conclusions. Are there specific outliers (or groupings of) that pop up frequently? Have further tests been conducted to verify why? Has there been an attempt to engineer out the variables? Follow these strings to ensure there are no loose threads – this is what the pharma due diligence team will do; be prepared for it.

DATA ACCURACY

When it comes to the information accuracy, particularly test results and conclusions, I strongly recommend you complete your own due diligence homework before going too far in any pharma negotiation (if not before starting any negotiations).

The first step is to verify that all original data, not just transcribed numbers, has been retained. Raw data is crucial. Just as the FDA will expect to be able to reconstruct tests and get similar results, so will a good pharma due diligence team. It is important to make sure that all results recorded are the actual results and that no rounding or averaging was involved.

Next, review protocol adherence. How accurately your testing plans were followed is something that will be examined by the pharma due diligence team. Failure to follow protocol calls into question the validity of any results. Key indicators that shortcuts may have been taken include the omission of bad results or a sudden pattern of errors, changes, re-dos, or cross-outs. During an audit, when I find what appear to be a series of errors, changes, or whiteouts, I look for other corresponding activities – change logs, equipment maintenance records, revised protocols, untrained lab technicians, and so on. If such records are missing, my suspicions increase. Documentation that explains changes or errors must be in existence and linked to the changes or errors.

Another accuracy aspect to preclinical and early stage clinical data quality is the degree of probability, or P-value, when it comes to whether any test results can be assumed to apply all similar conditions (rather than just the specific test conditions and subjects). The lower the P-value, the greater the confidence in the test result. Thus, a P-value of 4 per cent means the test and its results are statistically significant and a reviewer should have confidence that the test is a valid sampling. Just as medical journals typically will not publish papers touting results based on P-values higher than 5 per cent, the FDA will examine the P-value of test results for statistical validity, especially when it comes to determining whether a proposed new medicine is significantly better than current marketplace options.

There are many methods of determining the P-value. The weakest (in terms of overstating the degree of certainty) is the Wald Interval; the strongest is often NCSS LLC's exact double-binomial test. When it comes to reviewing test results, keep an eye out for reliance on the Wald Interval – this will be sure to draw third-party scrutiny. During an audit, if I find use of the Wald Interval, I ask the numbers be re-run using the harsher exact double-binomial test. Any test results that give a P-value greater than 5 per cent using the stricter exact double-binomial calculation should be approached with caution.

FINAL THOUGHTS

Late in 2007, the FDA published a new form (Form FDA 3674) for data certification of clinical trial data. This form is something that pharmaceutical executives have to sign. Draft up your own data integrity certification statement similar in format and language to the FDA's form, and then list out the items specifically reviewed. This type of signed summary is yet one more step in lowering perceived risks of research data quality.

The International Conference on Harmonization (ICH) has a guidance on Good Clinical Practice (ICH E6). Although these guidelines are for use in clinical trials, biotechnology executives looking to negotiate with their pharmaceutical counterparts can do themselves a favour by reviewing ICH E6. My clients have been able to adapt and apply several valuable insights from sections 7.3.5 on non-clinical studies and 7.3.6 on effects on humans.

Section 7.3.5 discusses ICH requirements for reporting the results of non-clinical studies such as toxicology tests, pharmacology and product metabolism studies. Section 7.3.6 lays out the requirements for known effects on humans as related to pharmacokinetics, safety, efficacy and other testing. Translating these written rules to preclinical work can be challenging, especially when it comes to knowing what to adopt, what to adapt and what to ignore, and so you may want to seek outside help preparing this for pharma negotiations.

Pharmaceutical executives sitting across the negotiating table are familiar with this guidance. The biotechnology executive who can speak with confidence that their data already meet those requirements will be that much closer to signing the best licensing deal.

Are you ready to turn compliance into competitive advantage?

© Cerulean Associates LLC