Navigating the Interesting and Strange Pre-Sub - Experience #3
Experience #3 - Issues We’ve Encountered
- Published on: March 20, 2020
In this third installment of our three-part series, we cover some anonymous but real-world examples of Pre-Sub issues we have encountered in 2019 and 2020. We discuss FDA’s inability to compromise on data that are necessary and Least Burdensome for a submission unless the studies are 100% of their own making—defeating the very purpose behind the Pre-Sub. We call it “It’s my way or the highway.” We also cover when they have backtracked on deals struck in Pre-Sub discussion—again defeating the very purpose behind the Pre-Sub. We call it “When the review team acts like Lucy and pulls up the football at the last second and Charlie Brown misses.” And among a few other smaller topics, how they unfairly edit meeting minutes to reflect points or matters not stated or discussed—again defeating the very purpose behind the Pre-Sub. We call it “He said, she said.” We conclude with key areas for FDA to fix.
Our continuing theme—the Addams Family
To explore the strange world of FDA Pre-submission meetings we use the metaphor of the Addams Family—the popular 1960’s TV comedy. You remember the characters—the patriarch and matriarch, Gomez and Morticia, who play the Office and Division Directors at FDA who in their running of the household are completely unaware that their family just doesn’t fit into the real world, that people are afraid of their ways, and don’t understand their intentions---which are often out of sync with the world outside their home. They live in this spooky mansion, Building WO66, filled with this odd cast of characters. The family gives sponsors a warm reception upon arrival, but they are thoroughly examined before entering and a sponsor walks through the halls with great trepidation holding a tense smile with unease not knowing exactly what is going to happen next, for the experience is strange and unfamiliar.
The idea behind this series is to alert the reader to issues that crop up in the use of the Pre-Sub (Q-Sub) program, identify areas of improvement (should the FDA read them), and help the reader anticipate and proactively address these issues in the course of their Pre-Sub.
It’s my way or the highway
If you have done a few Pre-Subs (we do 3-5 per month), you sometimes wonder why you do them because you quickly find that FDA is rarely wrong (just ask them), they hardly ever concede or falter in their position on what data are necessary, and they don’t honor process. Instead of critiquing the ideas given by a sponsor, FDA loves to dictate from scratch what they believe must be done. Their message is often delivered with certitude and a sense of almost imperialism. FDA reviewers are uninvited participants on the company’s development team. They are closet industry developmental team members, i.e. wannabies. They always feel they know better what needs to be done to establish substantial equivalence (SE) for a 510(k) or reasonable assurance of safety and effectiveness for a de novo or PMA. And industry’s opinions don’t matter, except as an opportunity for FDA to check the box to acknowledge industry has had a chance to express them.
While FDA’s attempts to create a package of data that will protect the American public is laudable and well-intentioned, their involvement often results in data requests involving over-sized parts and pieces, unnecessary to the creation of the data set to ensure a device is safe and functionally works. FDA’s requirements often substantially delay and many times kill innovations beneficial to patients. No matter what the quality and quantity of the data submitted by a sponsor, many reviewers seem to believe it is never correct, sufficient or adequate. Many FDA reviewers consistently ask for data that are scientifically interesting, but not required, to make a determination of SE or reasonable assurance of safety and effectiveness. And the scientific decisions are often divorced from the legal regulatory framework which circumscribes their decision making, i.e. the applicable standard for clearance, SE, or for approval, reasonable assurance of safety and effectiveness.
FDA has entertained the idea, for example, how to gather bench data and then allowed reviewers to continue to expand the boundaries of what data has been historically demanded. Many times, following the Pre-Sub, the review staff ventures outside of what was discussed and agreed upon in the meeting minutes. They often go outside of what previous companies had to produce for data and outside of FDA’s own guidance documents. Left unchecked, management often allows for those departures because, frankly, FDA is the beneficiary of ever-expanding data requests. This problem is exacerbated when large companies with the financial wherewithal and competitive desire to create barriers to market entry, agree to all sorts of testing that are frankly unnecessary or irrelevant to a clearance or approval. Small and medium-size companies must either acquiesce to similar testing or prevail in a challenge to the review staff. Over time, the overbroad requests of review staff continually expand what FDA can legitimately request of a sponsor because they can say it has been requested and provided before, albeit by a well-funded and often (anti-)competitively-motivated large company.
Frequently, a smaller company management, over our strong objection, wants to acquiesce to yet another bench test under a belief that “we are so close to finalizing this submission, let’s give the agency what they want even if it wasn’t part of the Pre-Sub discussion.” That sentiment proceeds out of a good faith belief by the company that it wants to go along to get along and close out this submission, but we see it for the potentially slippery slope it often becomes. Frequently, the reviewer finds the test raises increasingly granular questions which tickle the scientific curiosity of the reviewers and then a second and third test get requested, two of which have to be invented because there is no precedent for what the reviewers are requesting. And then our firm appeals to office management to arrest this never-ending cycle of reviewers who cannot pull the trigger on a submission.
FDA management created the Least Burdensome Red Flag (LB Flag) process
We have on a multiple occasions been politely chastised by upper management for failing to challenge sooner these never-ending requests for more and increasingly academic data. FDA management is quick to tell us that the issue should have been brought to their attention earlier, especially when revealed in a Pre-Sub. That is why they created the Least Burdensome Red Flag (LB Flag) process so these concerns over testing can be raised early before they are reluctantly (and usually over objection) agreed to and more and more irrelevant or unnecessary tests are required. We have had upper management tell us that they would have stopped the first extra test from being done, much less the third test. But once tests are conducted and results obtained they can give any reviewer enough ammunition to continue to drill down on new questions progressively being raised. We often get matters resolved quickly once we raise the issue to upper management.
What company management must understand is that most reviews at the staff level are heavily weighted toward risk analysis and the benefits of devices are often slighted for three reasons. First, because FDA’s view of industry are tainted, i.e. an FDA assumption of industry bias. Second, because FDA’s has an institutionalized risk-averse approach to conducting reviews. Third, because their requests feed the scientific curiosity of reviewers. Young inexperienced reviewers see boogeymen in every submission. They also see industry attempting to cut corners. And finally they see a sponsor submission as an opportunity to satisfy to their scientific curiosity and advance science. They are good at asking innumerable complex questions and rarely good at determining the adequacy of the existing data and sorting out what is sufficient to meet the SE (510(k)) or reasonable assurance of safety and effectiveness standards (for de novo and PMA) and in applying Least Burdensome requirements in doing so.
By waving the banner of patient safety, it seems as if FDA believes it is inoculated from concerns regarding the loss of jobs, intellectual property, and inability to obtain investment in medical devices (due to FDA’s unreasonableness and lack of predictability) matters seemingly too pedestrian for FDA to consider. No matter how much pressure is put upon FDA by Congress, patient advocacy groups and the press, many reviewers seem calloused or indifferent to the impact that their decisions have on the American patient and economy. But this is not an either/or proposition; we can protect patients, speed innovations to market and create U.S. jobs within the same regulatory system. We simply need to adjust the balance of risks with the benefits and ensure we are extending the benefits of new innovations to patients who need them and to the creation of jobs and support of medical device investment.
FDA needs to do a better job integrating law with science in decision-making in the Pre-Sub
FDA’s former Chief Counsel, Nancy Buc, speaking at the Fourth Annual Regulatory and Compliance Symposium in Washington, D.C. on September 30, 2009, provided terrific insight regarding the poor job FDA does in integrating law with science in decision-making (discussed in the context of the ReGen report) which still holds true today (emphasis added):
The second aspect of science that I want to talk about is the need for the rule of law and the role of law in agency decisionmaking about science. Again, I can take ReGen as my text. In that case, as far as I can tell from the recent report, the transcendent issue was not what the science told us about the performance (or lack thereof) of the device itself, but rather the standard to be applied by the decisionmakers. As the report outlines, some of the decisions about some 510(k)s are complex and complicated by uncertainties and inconsistencies in the standards – the legal standards – by which decisions are to be made about substantial equivalence. As I read the report, most of the problems stemmed from the fact that few if any of the decisionmakers at any level were willing to think through and articulate the standards by which decisions needed to be made. The absence of standards – and especially the absence of an articulated thought process – had everybody, within and without FDA, talking past each other.
Oddly, although the ReGen report discusses this issue at length, its recommendations do not explicitly include better law and better thinking about the law. Instead, they make “science” their first recommendation, as the current culture would want them to do. But if that wasn’t really the problem—if the problem was failure at many levels to think through the legal and regulatory questions and integrating them with the science – and if senior management was also doing science, just as the Branches were, then focusing only on science won’t help with the problem, because that wasn’t the problem in the first place...
One of the oddest things about FDA’s current practices with respect to science is the one-sidedness of it in a way that seems to me to be profoundly unscientific. I would think that science is best conducted where a proposal, a hypothesis, can be tested and vetted and debated and argued about by everyone with an interest. More and more, FDA seems to assume that industry people are not entitled to be fully part of that process because their views are tainted by their membership in industry. I have sometimes said to FDA people that an industry person is not wrong just because he or she is in industry, any more than an FDA person is right just because he or she is at FDA.
FDA’s current administrative practices pay lip service to Least Burdensome requirements
You are still entitled to argue in a Pre-Sub that the Agency’s request for data is not Least Burdensome, despite the Agency’s dismissive and obligatory attitude toward this statutory requirement. We typically pushback on the Agency with Least Burdensome arguments as a foundation for our more detailed arguments. FDA has become so risk-averse that it continually asks for data it wants, not what it needs, to establish SE or reasonable assurance of safety and effectiveness for a de novo or PMA. FDA has become so used to getting what it wants, it nonchalantly ignores industry pleas for reasonableness. In this mindless escalation of data requirements, statutory “Least Burdensome” requirements are also being ignored. Although FDA is politically astute enough to utter the words “Least Burdensome” in Pre-Sub meetings, it is at best a superficial utterance and often meant to placate applicants.
When the review team acts like Lucy and pulls the football up at the last second and Charlie Brown misses
We have had a number of Pre-Subs lately where FDA seems to negotiate in earnest, come to some compromise positions, and then pulls the football up just when our client is about to kick it. We have been in situations where the sponsor has detrimentally relied upon FDA’s continuing discussion of what and how to obtain the clinical and other data needed for clearance or approval.
In lieu of a prospective clinical trial, companies often offer to leverage real world evidence, gathered from a prospective protocol, and from sites that have used a device for a significant period. The data are either taken from U.S. sites where the device has been cleared for one indication, but is being used off-label for the new indication being sought. Or it is taken from European sites where the device was approved long before the U.S. clearance. FDA often assures our client in a Pre-Sub that they just need some confirmatory clinical evidence to support the long history of the predicate family, the bench testing provided, and the U.S. commercial complaint and MDR data or post-market surveillance data available from other countries (where the device was long ago approved).
The protocol is negotiated and memorialized in the Pre-Sub meeting minutes. FDA sprinkles in a few caveats here and there about the data that are about to be gathered, stating the sponsor must recognize this retrospective data may not be enough to substitute for prospective clinical data. But the review then continues with an encouraging tone (e.g., “You’re on the right track”) down the retrospective clinical data path. The review staff finalizes negotiating the parameters of the study and knows full well the extent of the effort about to be undertaken by the company to gather the retrospective data—in reliance upon FDA’s representations.
Then when the data are painstakingly gathered, according to the negotiated parameters, and the data are submitted, the review staff pulls up the proverbial football and states that either more retrospective data are required or a new prospective randomized controlled trial is merited, even when the retrospective data easily met the agreed upon success criteria.
In our numerous case studies, FDA’s significant reservations about a retrospective study, usually presented to the sponsor with great clarity at the 11th hour when the 180-day extension is about to expire, are frequently not communicated to the sponsor or their consultants before that time. The problem is one of omission, not commission. A review staff does not want to kill a submission outright or deflate a sponsor’s expectations, so they fail to be candid with a company. They do not say outright, “We do not believe what you are proposing will ever get you clearance or approval.” So, the sponsor marches along thinking they have FDA’s concurrence. The time and money are invested heading down a path that FDA already knows will not garner the clearance or approval they seek. It is a colossal waste of time, money and energy and ends up bankrupting small companies or results in tremendous and expensive delays.
What follows is that investors become spooked by FDA’s lack of transparency, predictability and unreasonableness. They either withdraw out of investor fatigue or frustration, usually both. And company management and their consultants lose credibility because they thought they had locked FDA down and had meeting minutes to show for it.
We think the review staffs often have a difficult time delivering an outright “no” to sponsors. They try to soften the blow and/or try not to sound dictatorial, uncompromising, not openminded, and/or not Least Burdensome. But they do a disservice to industry to not reveal the depth and the breadth of their skepticism and concern. If known earlier, the company could throw a Least Burdensome Red Flag (LB Flag) or do a formal appeal.
He said, she said
The last topic we wanted to cover is this disturbing trend of FDA failing to record with fidelity what was said at a Pre-Sub meeting. We also take issue with FDA’s ridiculous requirement that individual names not be recorded within meeting minutes. They also ask that the minutes be in outline form, not like a transcript, another ridiculous requirement that we almost never observe. We will cover all these objections below. Here is the situation analysis. From the sponsor’s side in preparation for the FDA meeting we assign two primary note takers, one from our firm who does this for a living, and one from the company. We ask everyone, however, to take notes, but we do not want our speakers saddled or distracted by taking notes. After the Pre-Sub meeting we have a debrief session, before getting on planes to go home, to recollect and record what everyone heard before they return to their busy schedules and memories can fade. Then the next day (often on the plane ride home) the lead note taker creates the draft minutes and everyone has an opportunity to edit them. If there are discrepancies, we get everyone on the phone to achieve understanding and consensus. It is a terrific discipline and works. The importance of recording the meeting minutes is so the purpose of the Pre-Sub can be fulfilled. We doubt FDA takes such a disciplined approach to creating their minutes—remember they are reviewing and editing the sponsor’s version, and often don’t provide their edits until almost 45 days after the meeting, when recollections can fade dramatically.
Let’s remind ourselves of the purpose for the Pre-Sub, i.e. to allow the two parties to dialogue about the potential regulatory pathway and the type and amount of data that will be needed for clearance or approval. It is also to memorialize what was said so both parties can look back to what was agreed upon. This gives sponsors the certainty they and investors need to move forward as expeditiously as possible with medical device development.
We are here to declare the Pre-Sub is failing of its essential purpose and for that, among many other concerns, FDA is not earning its user fees. We do a lot of Pre-Subs. In the early days we recorded what was said at a Pre-Sub and it was recorded accurately by us and our client sponsors and usually agreed by FDA without too much back and forth. In the early days we only had one meeting minutes disagreement filed with FDA. Today they are becoming more routine. Sometimes reviewers readily admit something wasn’t said in the meeting but confess that is their view/position on a topic and they want us to know that so they added it to the minutes. When challenged they pull it out. We fully appreciate that FDA wants a sponsor to know their position, even if not stated at the meeting, but the reason to bring it up at the meeting is so it can be discussed, debated and hopefully resolved. FDA’s failure to do so is undermining the very purpose for the Pre-Sub.
But what’s more disturbing is when FDA insists something was said, when it wasn’t. It is frustrating when FDA tries to add to meeting minutes things that were not said and conclusions that were not drawn because FDA has a duty to raise major issues in a Pre-Sub so that it can be discussed and debated. But, for whatever reason, discussion topics and comments are not being brought forward in a Pre-Sub, but there is an after-the-fact trend to add things that were not said. We know this is happening because unbeknownst to us (we know it is illegal) we’ve had some clients record conversations and then prove to us after-the-fact that FDA is not being truthful. We tell them to destroy the recording but there is living proof that FDA edits the minutes with things that were not said at meeting. But we also know that to be true just because of our ultra-careful manner in taking and creating the meeting minutes.
FDA must stop this practice and, more importantly, raise important issues at the Pre-Sub. FDA has 70 days (or 5 days in advance of a meeting, whichever is soonest) to review the Pre-Sub and prepare a written response. That is sufficient time for FDA to be fully able to articulate their position and the reasons why. In contrast, industry typically has only 5 days in advance of a Pre-Sub meeting to prepare their response.
Related to that concern, but it will not be addressed at length here is the liberty FDA takes to defer discrete and insular topics to future, often multiple, Pre-Subs. It is enough that Pre-Subs came on the horizon to replace FDA’s inability to facilitate product files to conclusion. The Pre-Sub was supposed to help FDA and industry come to agreement upon the regulatory path and data. But the new normal is for FDA to ask for multiple Pre-Subs. FDA defeats the spirit behind user fees by buying themselves more and more time pre- and post-submission to make a decision to approve or clear a device. FDA, for example wants multiple Pre-Subs just to review and “approve” an IDE before it is actually submitted for formal approval. The user fee timelines have elongated to a frustratingly disappointing extent and yet FDA gets ever more appropriations and increased user fees to reward its poor performance. But that is a topic for another day.
Let’s move to the unnecessary requirement that FDA tries to dictate to industry how the meeting minutes should be constructed, despite our First Amendment right to free and truthful speech. FDA wants no names recorded, but that everyone on FDA’s side be recorded as “FDA said.” Recording the names of those who make comments does three things. First, it tells us the importance of the commenter. A comment by an office director has more experience and weight than if by a first-year reviewer. A chief medical officer might have more weight than other colleagues. Second, it really helps everyone remember what was said and when and why it was said. It gives color, context and clarity to the conversation. If for example, if a biostatistician took an epidemiological position on a debate point and the chief medical officer disagreed with his or her own colleague from a common-sense medical perspective, it would be irresponsible to not record the disagreement within the review staff. It is especially important for the company to do when the chief medical officer’s position is aligned with the company’s. Third, if the sponsor decides to appeal to upper management, the review staffs are acutely aware that their own words may later be used against them and they should, because they should not change their position once taken.
Let’s face it, review staff does not like to commit to anything collectively or individually, which also infects why they are afraid to commit to anything which is later recorded in meeting minutes. But their lack of courage, their inability to make a binding decision, is fundamentally antithetical—once again—to the purposes behind a Pre-Sub process. They want to be free moral agents—seemingly open-minded and gregarious in a Pre-Sub—but loathe to commit to anything, so they can keep their options open. Industry needs just the opposite of that and that is why the Pre-sub process was created in the first place, i.e. to force FDA to commit and achieve certainty for the sponsor.
FDA also now wants Pre-Sub meeting minutes to be in outline form rather than in what they pejoratively call a “transcript.” But if one were to think again to the purpose of the Pre-Sub it was to understand what was agreed upon and the rationale for that agreement is vital and often you cannot come close to capturing it in outline form. And it shouldn’t matter one way or another. If one firm wanted to capture what was said in outline form and another in a narrative, i.e. transcript, they should be able to do what they want.
The bottom line is it is rarely about what works for the customer, i.e. the sponsor, patient, physician, taxpayer, or the process; it’s mostly about FDA’s agenda and what makes their life easier. The focus should be on Least Burdensome practices which can expedite new innovations to market.
FDA’s user fees are based upon their performance and we contend the Pre-Sub, while still a good idea, is starting to show its cracks. FDA needs to ask itself is the Pre-Sub failing of its essential purpose. FDA management needs to revisit their performance, but please don’t issue another guidance document, just fix what you have.
Key areas for FDA to fix include:
Train reviewers to make risk-based decisions in support of the actual standards. Substantial equivalence is substantial not complete equivalence and is established in a comparative sense. Reasonable assurance of safety and effectiveness requires reasonable assurance not absolute assurance and is established in an independent sense.
Stop allowing de novo to be an outlet for the 510(k) program when it was supposed to be an outlet for the PMA program. This is another example of FDA not engaging in risk-based decisionmaking. In addition, the 510(k) definitions are more flexible and accommodating than the way FDA interprets them; too many submissions are pushed off the 510(k) pathway and onto the de novo pathway.
Require FDA to disclose substantive issues and supporting rationale in their written response and at the Pre-Sub meeting, i.e., be candid. Omitting this input is akin to false and misleading advertising. When disclosure is made it is not enough for FDA to have a “gut impression” or intuition on issues, they must be data-driven as they expect industry to be.
Demonstrate commitment to Least Burdensome requirements through decision making that reflects use of non-clinical performance data and sensible amounts of clinical data, both of which are meaningful and useful to demonstrate SE and reasonable assurance of safety and effectiveness.
Get reviewers in the mindset of critiquing what they’ve been given, not taking the initiative to create a new plan out of whole cloth.
Treat a Pre-Sub assessment as you would an actual premarket submission - would FDA make the same decision if the data provided in the Pre-Sub were submitted as a premarket submission? Pre-Subs are not a license to take over the plane and fly forever and wherever you want, they are meant to prepare a solid flight plan for a specific destination.
If FDA changes the flight plan in the middle of a flight (pulls a Lucy), industry should get a full refund of their ticket price (i.e. the user fees paid for the resulting commercialization submission). Wishful thinking.
Limit the number of Pre-Sub dress rehearsals. The potential endless cycle of Pre-Subs and quasi-commitment means the public will never get to see the end result.
Require FDA to provide metrics on the percentage of premarket submissions that were landed (cleared or approved) consistent with the Pre-Sub flight plan. Include time to clearance/approval metrics based on the presence or absence of a related Pre-Sub to provide transparency to the value of holding a Pre-Sub.