Chasing a Cure, Part II, evaluating legitimacy

In Part I, I laid out some of reasons why finding a cure for cancer is a really big deal. A REALLY BIG DEAL. So many people’s lives are riding on medical innovation and creativity, mine included.

And so it’s important to be able to evaluate whether a “source” or a “study” is something to rely on. So many times, I see people posting articles or literature reviews and calling it support or a study. And so I think it’s important to review what to look for when we look for support for a position.

Generally and overall, some common evaluation criteria include: 1) The purpose and intended audience; 2) authority and credibility; 3) accuracy and reliability; 4) currency and timeliness; and 5) objectivity or bias. Each of these elements can look different when evaluating varied sources, but here’s a general analysis.

1. The Purpose and Intended Audience. A study that has been completed by a trained scientist, evaluated by peers, and intended for scientists is the best example of “proof” that a concept or treatment can be replicated. For a treatment or “cure” to be valid, it can’t just work for one person or for a small group of people and so there are rigorous standards for research to even be published, which protects all of us.
2. Authority and Credibility. Who is doing the research, their credentials, their training, etc., is a really big deal. The scientists who are doing the research on the treatments that are keeping those of us living with cancer alive are doing so after years and years of study and training and mentoring.
3. Accuracy and Reliability. I’m not a statistician, but there are a lot of really smart people who review the data and numbers in a study to ensure the accuracy and reliability of the data in a published study. Again, to be valid, to be reliable, a study has to work for more than just one or a small group of people.
4. Currency and Timeliness. There are so many scientific questions posed all the time and most are either not answerable or negative. In fact, I often wonder how so many scientists remain focused and enthusiastic when the answers are so often no or not yet. For a study to be helpful, the questions and answers need to be timely and helpful; many are not, many are waiting for the next step or level or application.
5. Objectivity or Bias. No study is complete without acknowledging bias and addressing the opposite perspective or the roadblocks or the negatives. Every study must acknowledge these issues openly and address them. An article that simply presents one point of view and then continue from there is not an article that is examining an issue from all sides. To make an argument, one must anticipate the perspectives of others to test one’s opinion/hypothesis.

Here’s some additional specifics …

• Look for a well-respected publisher. It’s so important to ensure that the source is from a reputable publisher who actually checks the accuracy of the information. On the internet, look for .gov (a government site) or .coms and .orgs. The National Institute of Health (NIH), National Cancer Institute (NCI), New England Journal of Medicine (NEJM) are all reputable sources. Think about The NY Times or the Wall Street Journal too, the sources that are recognized. If you’ve never heard of the website or source, look further.
• One thing that these reputable sources have in common is that the articles and information is peer reviewed, meaning that professionals other than the author reviewed the information and agreed that the data can be relied upon. Look for a statement that an article or source has been peer reviewed, this is important.
• Citations for sources used — this is so important, watch for statements that have no sources or support. If an article is reputable, then there are other sources that support the conclusions, other articles that build upon or utilize the same information.
• If the author includes a way to contact them — while I realize this is not scientific, I believe that authors who are pursuing knowledge are open to hearing from others and want to engage in healthy debate. When an author doesn’t allow this opportunity, it makes me pause and consider whether I can trust that author.

At the end of the day, knowing what to rely upon can be so hard and there are so many pieces of information out there that we want to rely on because we’re all looking for the cure that we need. False hope, though, is not preferable to no hope.