Is there more to conflict of interest than money?

We take a look at whether ideological incentives can create conflicts of interest, using Veterans Affairs health care research as a case study.

If we were to tell you an accomplished researcher conducted a study with results that were advantageous to them, what would you think of?

One situation that might come to mind is an academic, pressured to “publish or perish” in a world where an ever growing list of publications makes their CV look better. In some cases, the actual results of the study aren’t as important as the fact that the study was published at all. In other cases, more exciting results can lead to more attention and research citations, making researchers and medical journals more likely to publish results with interesting conclusions or ones that challenge the status quo. The end result here is publication bias, where boring or less interesting results tend to go unpublished.

Alternatively you might think of someone studying a drug or a medical device made by a company that they have a financial interest in. If the study shows the drug works well, that person may stand to gain financially, creating potentially concerning incentives, perhaps subconscious, to make the drug look as good as possible. This potential threat to objectivity is a conflict of interest, or COI, and it’s these types of financial COIs that tend to get the most attention—not just in research, but in business, government, and other arenas where financial incentives could create COIs.

But researchers have plenty of motivations beyond citation counts or making money. The way we view the world, our ideologies, can impact the questions we ask but may also impact the answers that we find. For example, would a researcher who believes in or has advocated for a particular health reform (e.g., the Affordable Care Act) be likely to find negative effects of the reform? Alternatively, would someone who advocated strongly against such reform be likely to find positive effects of the reform in their own research? Many questions in health policy tangle with politics and ideology, ranging from the role of government versus the private sector in health care, issues around reproductive care, drug prices, malpractice, the list goes on. Our ideologies pose a strong, potential conflict of interest that often goes unnoticed, in part because the issue is hard to study.

Now, most of us get into research because we genuinely want to explore and understand the world to make it better. We certainly want to feel like we’re meeting our individual and institutional missions of doing impactful work, and we’re inclined to want to prove that to others as well. But what happens when we are doing research about ourselves, our colleagues, our field? When we’re doing research about ourselves, it creates a conflict of interest, too—one about prestige and recognition, but a COI we don’t always think about.

The Case of the VA

The U.S. celebrated Veterans Day this past week, where we paused to recognize and show gratitude for the work and sacrifice of our neighbors who have served in the military. But another way the country thanks our veterans is by providing them with a variety of benefits through the Department of Veterans Affairs, or the VA. Health care provided through the Veterans Health Administration is one of those benefits, and there’s a large network of VA hospitals and medical offices across the U.S. that provide care to our veterans.

Many people who work in VA health care (Chris worked at VA Boston for four years as a resident and chief resident) will readily acknowledge that the system has problems. They work hard to do as much as they can for the veterans they serve despite the VA’s shortcomings. The same can be said for health care workers in private and public sector organizations outside the VA: most of us acknowledge that our health care system has some serious problems, but we do our best to care for the patients we serve.

Both the VA and the rest of the U.S. health care system are failing patients, in some way or another, every minute of every day. But the VA faces additional scrutiny. When the government is tasked with providing health care to this special population of patients, it does so with additional attention from veterans, the general public, and congress alike. As such, much attention has been given to the shortcomings of the VA health care system over the years, even though research has repeatedly shown that medical and surgical care at the VA is generally as good or better than care provided in the rest of U.S. health care.

This got us wondering: If you were a VA-based researcher studying whether VA care was better or worse than non-VA care, would you be incentivized to make the VA look good? “Clear your name,” so to speak? There’s an institutional, and possibly ideological, conflict of interest here. Could that institutional COI lead to biased research reporting?

“Spin” Doctor, or just Doctor?

In a study with orthopedic surgeon Nathan Varady, we wanted to see whether institutional conflicts of interest might bias research reporting in a way that reflects positively on the institution. The VA presented us with an opportunity to study this, since we can look at the authors of studies comparing VA care to non-VA care and we could easily determine if they were affiliated with the VA or not.

We wanted to look at two things. First, we wanted to know whether VA-based researchers more likely to publish studies that portrayed the VA in a good light when they were comparing VA-care to non-VA-care than they otherwise would. Second, we wanted to know if VA-based researchers were more likely to use language to “spin” their results in a way that shed the VA in the most positive light.

We started with a set of 68 research abstracts of studies comparing VA care to non-VA care. We then looked at other recent publications by the same researchers that didn’t compare VA care to non-VA care—that way we knew the types of results they tend to study and the type of language they tend to use. We took a look at the results and the language manually, and we also ran it through Google’s natural language processing tool that gave us a “sentiment” score, ranging from -1 (“clearly negative” sentiment) to +1 (“clearly positive”), with 0 being neutral. Here’s what we found:

• VA-based researchers were actually less likely to publish positive studies (“positive” here meaning “good”) and more likely to publish negative studies about VA care as compared to when they studied other topics.

• Whether they were studying VA care or other topics, researchers were equally likely to “spin” their abstracts to sound better when reporting their results or conclusions, such as by making definitive statements rather than words like “may” or “suggest,” or by failing to mention significant limitations in their abstract.

• When sent through Google’s language AI, the sentiment of the language was equally neutral when researchers were studying VA care as compared to other topics.

In short, the institutional COI wasn’t leading these researchers to make the VA look better when reporting their research—and they may have actually been biased in the other direction!

Case closed?

Our study of these VA researchers was but a snapshot of the large and complex topic of institutional and ideological COI. The VA study looks at issues of institutional pride and reputation at the VA, which doesn’t compete in the health care marketplace to the extent of other organizations.

Research institutions create all kinds of incentives that create COIs for researchers that are not directly financial and that wouldn’t be reported on typical COI disclosures. Other institutional COIs could arise from financial incentives for the institution, rather than individuals, to receive funding for its mission. For example, a charitable foundation needs to show it’s helping patients to continue to receive donations—but no individual researcher would need to report this as a COI.

While COIs of some sort are all but unavoidable, just because there is a COI or a potential COI present, it doesn’t mean that a research study’s findings are necessarily biased. For example, it’s quite common to see high-quality published research funded by drug companies about their own drugs because steps are taken to mitigate the inherent conflict of interest. So what’s most important is to be aware of additional potential COIs when evaluating a research study—looking beyond simply the reported financial disclosures.