Chasing a Cure, Part II, evaluating legitimacy

In Part I, I laid out some of reasons why finding a cure for cancer is a really big deal. A REALLY BIG DEAL. So many people’s lives are riding on medical innovation and creativity, mine included.

And so it’s important to be able to evaluate whether a “source” or a “study” is something to rely on. So many times, I see people posting articles or literature reviews and calling it support or a study. And so I think it’s important to review what to look for when we look for support for a position.

Generally and overall, some common evaluation criteria include: 1) The purpose and intended audience; 2) authority and credibility; 3) accuracy and reliability; 4) currency and timeliness; and 5) objectivity or bias. Each of these elements can look different when evaluating varied sources, but here’s a general analysis.

  1. The Purpose and Intended Audience. A study that has been completed by a trained scientist, evaluated by peers, and intended for scientists is the best example of “proof” that a concept or treatment can be replicated. For a treatment or “cure” to be valid, it can’t just work for one person or for a small group of people and so there are rigorous standards for research to even be published, which protects all of us.
  2. Authority and Credibility. Who is doing the research, their credentials, their training, etc., is a really big deal. The scientists who are doing the research on the treatments that are keeping those of us living with cancer alive are doing so after years and years of study and training and mentoring.
  3. Accuracy and Reliability. I’m not a statistician, but there are a lot of really smart people who review the data and numbers in a study to ensure the accuracy and reliability of the data in a published study. Again, to be valid, to be reliable, a study has to work for more than just one or a small group of people.
  4. Currency and Timeliness. There are so many scientific questions posed all the time and most are either not answerable or negative. In fact, I often wonder how so many scientists remain focused and enthusiastic when the answers are so often no or not yet. For a study to be helpful, the questions and answers need to be timely and helpful; many are not, many are waiting for the next step or level or application.
  5. Objectivity or Bias. No study is complete without acknowledging bias and addressing the opposite perspective or the roadblocks or the negatives. Every study must acknowledge these issues openly and address them. An article that simply presents one point of view and then continue from there is not an article that is examining an issue from all sides. To make an argument, one must anticipate the perspectives of others to test one’s opinion/hypothesis.

Here’s some additional specifics …

  • Look for a well-respected publisher. It’s so important to ensure that the source is from a reputable publisher who actually checks the accuracy of the information. On the internet, look for .gov (a government site) or .coms and .orgs. The National Institute of Health (NIH), National Cancer Institute (NCI), New England Journal of Medicine (NEJM) are all reputable sources. Think about The NY Times or the Wall Street Journal too, the sources that are recognized. If you’ve never heard of the website or source, look further.
  • One thing that these reputable sources have in common is that the articles and information is peer reviewed, meaning that professionals other than the author reviewed the information and agreed that the data can be relied upon. Look for a statement that an article or source has been peer reviewed, this is important.
  • Citations for sources used — this is so important, watch for statements that have no sources or support. If an article is reputable, then there are other sources that support the conclusions, other articles that build upon or utilize the same information.
  • If the author includes a way to contact them — while I realize this is not scientific, I believe that authors who are pursuing knowledge are open to hearing from others and want to engage in healthy debate. When an author doesn’t allow this opportunity, it makes me pause and consider whether I can trust that author.

At the end of the day, knowing what to rely upon can be so hard and there are so many pieces of information out there that we want to rely on because we’re all looking for the cure that we need. False hope, though, is not preferable to no hope.

8 thoughts on “Chasing a Cure, Part II, evaluating legitimacy

  1. This is truly important information for people to have, understand and follow. Thank you for delivering this to your audience. You’re doing a great service to the world.

    Liked by 1 person

  2. Well done, Abigail. I was a medical writer and editor for a number of years, and you touched on all the important points. They’re not foolproof; sometimes even the most reputable researchers have been found to have been influenced by greed, but they’re very useful.

    Liked by 1 person

    1. Right?! Always makes me think twice if an author isn’t open to feedback. Goes back to the ability to look at a topic from all sides, not just “yours.” If you can anticipate and address issues, then you haven’t really thought the topic all the way through. In my opinion.

      Like

  3. Excellent questions 🙂 Following my diagnosis when I started to read all kinds of stuff relating to cancer and its treatment, I found ‘Bad Science’ from Ben Goldacre very useful too, as it made me look at the basics of any study – what questions were being asked, who made up the study etc. As an example of what I mean – I suffer from ha fever so take anti-histamines, but I seriously struggle with drowsiness. A particular brand developed a sub-brand which was aimed to limit drowsiness and, in my experience, it managed to retain its efficacy against the symptoms of hay fever, while not rendering me a zombie. But a study testing its efficacy was carried out containing almost no participants who struggled with the drowsiness aspect of anti-histamine drugs, so the medication wasn’t green lighted for widespread use. Fortunately I have an enlightened doctor. But honestly, why would you set up a study this way?

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s