When Science becomes Propaganda

By Peter Laird, MD

The advancement of science hinges upon the tenants of truth and honesty engendered in the scientific method establishing the randomized controlled trial as the gold standard to eliminate bias, so keeping the results impartial from researchers personal preconceptions. In many ways, science depends implicitly upon the trust earned by those that practice these methods as much as the method itself. The veracity of a study is thus dependent upon the personal and professional integrity of the researchers themselves as well as those that publish their results. Today, with stiff competition among a myriad of medical journals, it is not surprising to see shades of truth stretched in an attempt to steal more readership. Yet, in the end, it must be avoided at all costs to maintain integrity and trust in that journal which is much more damaging in the long term than what the small temporal gain of a one time sensational headline may bring.

It is my opinion that ISN stepped over this fine line of scientific impartiality into journalistic sensationalism in their recently titled article: Nocturnal hemodialysis does not improve overall measures of quality of life compared to conventional hemodialysis. The title suggests a shocking reversal of numerous observational studies depicting exciting benefits of nocturnal dialysis. Yet, the authors themselves make no such claim in their concluding remarks:

There were limitations to this study and the results should be interpreted within the context of these limitations. First, the study was powered to detect changes in left ventricular mass, and thus statistical power was inadequate to detect clinically significant differences in several of the quality of life outcome measures. For example, we would have needed 342 patients per group to have 80% power to show a 0.03 unit change in EQ-5D index scores (the minimally clinically detectable difference). . .

In conclusion, our primary quality of life analysis did not demonstrate a statistically significant change between nocturnal hemodialysis and conventional hemodialysis. However, clinically important and statistically significant changes in some kidney-disease-specific measures were observed during the trial, and appeared durable over time. The magnitude of differences in EQ-5D index scores observed in secondary analyses (if confirmed in adequately powered randomized trials) would be of substantial clinical significance. These findings support the increased use of nocturnal hemodialysis in selected groups of patients.

In my opinion, the title suggests a completely discordant conclusion than the actual summation by the authors who stand in support of nocturnal dialysis based on their findings in this study that suggested, but did not prove due to too small a sample size, that some quality of life measures did in fact improve with nocturnal dialysis. Bill likewise commented on this study in an earlier post after the abstract was first released. I am saddened that the negative and unsupported title of this indeterminate and underpowered study may impart a lasting negative impression upon those that do not take the time to review all of the evidence presented by these authors.

In my opinion, the title is so contradictory to the evidence presented that I must question the credibility of the editorial staff who have the final authority over the title of an article as well as its content. For science to survive in the twenty first century, old fashioned values of impartiality and fidelity must be maintained and promoted actively. I am once again troubled by the negative and unsupported title of what was otherwise, in my opinion, a well researched and well written article. It is time to return to truthfulness in all parts of scientific reporting and avoid the sensational tabloid reporting methods reserved for torrid Hollywood gossip papers. In my opinion, it is not fitting to find examples of Hollywood sensationalism in medical journals devoted to finding the truth of important and lasting issues especially when the authors themselves support and found evidence suggesting substantial clinical significance in support of nocturnal dialysis. Simply reading the title of this article lends to a completely contrary opinion than that intended by the authors concluding remarks.

I would hope that ISN would take notice and correct this, at best, ambiguous title in favor of a more accurate first impression for this inconclusive and underpowered study that actually supports nocturnal dialysis by the findings of trends in improved quality of life indicators. The issues at hand demand honesty in all aspects of reporting and most importantly that which gives the first and most lasting impression, the title itself.

Comments

[Miriam Lippel Blum]

Just want to say that I participated when Rand Corp. developed their "Quality of Life" instrumentation. I was a PD patient then and they were looking for renal patients to test their new questionnaire and give feedback, so I agreed. I gave them an earful about how their criteria were poor and most of what they asked about really didn't say too much about life quality and therefore, wouldn't give results that had much credibility if they were really looking for truth. I saw the final form...and it was clear they didn't take what I said into account. I was not impressed with what was passing off as scientific standards or integrity.

But they gave me a coffee mug. I guess they thought that would improve my quality of life.

[Michael Fraase]

If I was a betting man, I'd lay good money that a marketeer of some flavor wrote the headline, not the author or editor of the article. It's happened to me more than once.

Also, I'd like to add the tenant of *transparency* to your laundry list of what advances science. I'm going to be writing more about its importance later today about the launch of Medpedia.

[Qais Al-Awqati]

Please allow me to correct certain misconceptions here. It is true that the Editors of Kidney International have the last word on titles and they change them to be more consonant with the major findings of the paper. We then send the changed Title and Abstract to the lead author, in this case Dr Braden Manns who made additional changes to finally come up with the title that he approved. After Dr Manns approved the Title and Abstract the paper was sent to the copy editors who suggest additional changes in the main text. The title and abstract and the contents of the paper are a result of a dynamic process where the authors, the anonymous reviewers and the editors suggest changes and respond to the critiques by changing the text to come up with the final version. In no case are the Title, Abstract or contents determined by anyone without the final agreement of the authors. Hence I believe that your ire or skepticism is rather misplaced.
Qais Al-Awqati
Editor in Chief
Kidney International

[Peter Laird, MD]

Dear Qais Al-Awqati,

I believe that you are missing the essence of my opinion on your article and your editorial duties as the final approving authority for all content in your journal. For anyone that reads the concluding remarks and then compares that to the title, they are completely discordant. Further, the title suggests a completely negative outcome with proof of that outcome that does not exist. The trends in some of the parameters were not against nocturnal dialysis as your title declares emphatically, but simply did not have enough power in this study to reach the opposite that your authors adctully draw with the conclusion of another postitve confirmation of improvements of quality of life with nocturnal dialysis that your authors correctly note:

The magnitude of differences in EQ-5D index scores observed in secondary analyses (if confirmed in adequately powered randomized trials) would be of substantial clinical significance. These findings support the increased use of nocturnal hemodialysis in selected groups of patients.

How do you as an editor resolve the conclusion of your author above who is in support of nocturnal dialysis based on the findings of his study with the final emphatic title declaring a negative outcome for nocturnal dialysis quality of life? Nocturnal hemodialysis does not improve overall measures of quality of life compared to conventional hemodialysis.

It is my continued opinion that those who simply read the title of this article would find this same discord between title and the content of your article that I did. I believe that the editorial staff does have the final authority for all works published in your journal to represent accurately the content of an article thus sharing the greater duty than even that of the authors. It is my opinion that this was not accomplished with the article in question above.

It is my further opinion that this type of sensational title in complete opposition to dozens of prior articles on nocturnal dialysis could be misleading at best and thus could be used by opponents of nocturnal dialysis simply by virtue of the title without regard to the content which did in fact trend towards support of nocturnal dialysis but did not have the power or size to reach that conclusion.

In my opinion, I must simply question whether it was appropriate for KI to approve such a controversial title usually reserved for landmark studies that reverse the findings of prior studies when your study does not in the least reverse those findings.

It is simply a study too small and underpowered to conclude any reversal of prior studies on nocturnal dialysis in contradistinction to previous findings of significant benefit in other studies. It did not in fact prove anything, positive or negative because it is an underpowered study. The best that we can say that further study is warrented with a larger study group of sufficient power to confirm the trends of this study, which your authors so concluded.

I was quit surprised to see that in the end of my review, there was no proof of such a controversial claim because your small study lacks the power to so declare a reversal of all prior postive outcome studies from nocturnal dialysis that your title emphatically implies. In stated fact, the authors found trends supporting an improvement of quality of life in direct opposition to the final title.

I appreciate your input on the editorial and review process for all of your articles as well as final approval of the authors which is of paramount importance in my opinion. Nevertheless, in my opinion, in the case of the article in question, the final product of that process did not represent the content accurately and could, in my opinion, be considered sensationalism which I would assume by the statement of conclusion, was not the author's intentions.

I stand by my opinion that medical journals should overcome the need for sensational titles to compete against other journals readership share, and do what they are charged to do, report the truth accurately and without bias. That is what science represents.

It is my continued opinion that KI failed to keep their fuduciary duty to the tenants of science to avoid bias by virtue of such a contradictory title unsupported by facts in the content of the article. I would hope KI and ISN would consider correcting this discordancy by rewriting the title at this time to better reflect the true content of this article in a more accurate fashion.

[Zach]

Interesting "Viewpoint" article on this very subject from the June issue of "Nephrology News & Issues:"

http://www.nephronline.com/article.asp?IndexID=171

[Peter Laird, MD]

Zach, thanks for the link, I was not aware of the editorial. I believe that Dr. Mendelssohn spoke very eloquently on the issue at hand. Truth and integrity are the hallmark of science and must be protected at all costs.

I just wish to thank Bill for letting me be a part of DSEN. I have learned much in the last year from Bill and my main goal on any article is to get past his red editing pen if possible. I stand firm in supporting Bill and his long lasting advocacy for not only truth in dialysis, but most importantly equality of health based on the highest standards we know.

[Bill Peckham]

I wish he would have properly credited you as the author Peter and it is routine to link to a referenced article.

I applaud the sentiment. Thanks for the link Zach.