False Claims Of Evidence-Based Practice in Psychotherapy

The term “evidence-based” therapy is currently being used by many in the field of psychotherapy, including practitioners and marketing personnel, to refer to something quite different from what a reasonable person might otherwise assume, and is therefore, misleading and problematic. The goal of this article is to create awareness of how such false narratives of misleading and simplistic evidence-based claims are creating wedges, and perpetuating the already underlying turf-wars between different psychotherapy models and theoretical orientations. I hope this article will unify all practitioners and researchers in the field of psychotherapy, irrespective of the psychotherapy models or value-base they identify with, to band together to call for an Evidence-Based framework that better echoes the complexities of the actual practice of psychotherapy, in order to move towards the goals of true Evidence-Based Psychotherapy Practice (EBPP).

Introduction

In the Evidence-Based medical model, the gold standard for evidence of effectiveness is the Randomized Control Trial (RCT). This was adopted and used to study the effectiveness of various psychotherapy models being practiced as well. Renowned and respected clinician, John Marzillier had pointed out, in as early as 2004, that these RCT studies do not translate so well to measure outcome results in psychotherapy and lists the following problems:

  1. Classifying subjects on symptoms or psychiatric diagnosis due to the inherent problems associated with using a medical model for diagnosis. In addition, patients’ problems are not solely defined by their “symptoms” or “diagnosis”, and therefore treatment cannot be determined without taking the person’s personalities, experiences and circumstances into consideration.

  2. The limitations of outcome research using RCT studies, and

  3. Therapy being essentially a personal relationship

on why using the results of RCT studies alone to make claims of effectiveness is flawed and misleading. In fact, the better designed the research study using RCT (manualized scripts, frequent assessments, recordings of the process), the more distanced it is from actual clinical work. This might seem pretty obvious to practitioners of psychotherapy: an Evidence-Based medical model does not meet the demands and needs of Evidence-Based psychotherapy due to the complexities and interplay of personal and inter-personal variables in the actual practice of psychotherapy.

Marzillier (2004) eloquently states his position as follows:

"Evidence-based psychotherapy seems a reasonable aspiration. There are many weird and wonderful treatments in the field; surely it makes sense to know whether or not they work?

[...] Yet I want to argue that evidence-based psychotherapy is a myth.

I am not against scientific research in psychotherapy. On the contrary, I believe that scientific advances in psychology and related disciplines are important to the development of psychological therapies. But that is different from claiming that what psychotherapists do is, or should be, securely founded in evidence of effectiveness; that, for example, we can say with authority that depressed people are most likely to benefit from cognitive therapy or that research has shown that 90 per cent of people with panic attacks will recover with anxiety management. Such claims are in my view misleading and simplistic, and it is this ‘outcome research’ I have a problem with. It does justice neither to the complexity of people’s psychology nor to the intricacies of psychotherapy." (p.392)

The call and support for Evidence-Based Psychotherapy Practice (EBPP) made by the American Psychological Association (APA) in 2005 was to ensure that psychotherapy practices reflect clinical judgement, patients' values and preferences, and relevant scientific research. “Evidence-based practice in psychology is the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences.” ("APA Task Force on Evidence-Based Practice," 2006). By not limiting the evidence from relevant scientific research to the outcome studies using RCT studies alone, and by including the need to reflect clinical judgment (i.e., therapist expertise) and patient's values and preferences, in principal, EBPP seemed to take into consideration the concerns outlined by Marzillier (2004).

An Evidence-Based medical model does not meet the demands and needs of Evidence-Based psychotherapy due to the complexities and interplay of personal and inter-personal variables in the actual practice of psychotherapy.

Research based on empirical studies shows that the quality of the therapeutic relationship is as important or more important than techniques.

All of the above point to the need for EBPP to move away from the reliance on RCTs to scientific methods and studies more suited to psychotherapy (e.g., empirical research that includes quantitative and qualitative studies, practice-based evidence, and a focus on more person-centered outcome measures), that can balance the study and effect of interventions, as well as parameters of the therapeutic relationship, and the interplay between them. However, the field of psychotherapy continues to stay in the clutches of the medical model and the use of RCTs in which techniques dominate at the expense of personal and inter-personal variables despite the considerable evidence against it.

Why is that?

Evidence-Based Psychotherapy Practice (EBPP) vs. Empirically Supported Treatment (EST)

A treatment qualifies as an Empirically Supported Treatment (EST) based on successfully replicated, randomized control trial (RCT) studies. Prior to the publication of the “APA Task Force on Evidence-Based Practice” (2006), the term Evidence-Based psychotherapy was used interchangeably with EST, for example, as done by Marzillier (2004) quoted in the introduction. The APA Task Force on Evidence-Based Practice has clearly delineated the differences between EST and EBPP as quoted below:

It is important to clarify the relation between EBPP and empirically supported treatments (ESTs). EBPP is the more comprehensive concept. ESTs start with a treatment and ask whether it works for a certain disorder or problem under specified circumstances. EBPP starts with the patient and asks what research evidence (including relevant results from RCTs) will assist the psychologist in achieving the best outcome. In addition, ESTs are specific psychological treatments that have been shown to be efficacious in controlled clinical trials, whereas EBPP encompasses a broader range of clinical activities (e.g., psychological assessment, case formulation, therapy relationships). As such, EBPP articulates a decision-making process for integrating multiple streams of research evidence—including but not limited to RCTs—into the intervention process.” (2006).

So what does this EBPP look like?

“How to improve psychotherapy outcome? Follow the evidence; follow what contributes to psychotherapy outcome. Begin by leveraging the patient's resources and self-healing capacities; emphasize the therapy relationship and so-called common factors; employ research-supported treatment methods; select interpersonally skilled and clinically motivated practitioners; and adapt all of them to the patient's characteristics, personality, and worldview. This, not simply matching a treatment method to a particular disorder, will maximize success." (Norcross and Lambert, 2011, p.13).

The label of "evidence-based" has however been appropriated by proponents of mainly brief and manualized therapy models based solely on outcome research using RCTs. It is being widely asserted that these models are scientifically proven and superior to other forms of psychotherapy, which is a false narrative (Shedler, 2015). In other words, ESTs are claiming the term “evidence-based” therapy, when in reality the term Evidence-Based Psychotherapy Practice (EBPP) refers to a more comprehensive concept.

Although researchers and practitioners have questioned and challenged the use of RCTs in making such claims, and empirical research does not support these claims, these false narratives are being maintained and perpetuated by the proponents of these models. It is also being used to exclude other psychotherapy models that have a different value-base, from being considered legitimate forms of therapy.

I will use the policies of Lyra Health, a new healthcare startup company, to illustrate this. Their selection criteria for psychotherapists explicitly excludes any psychotherapist that works from an attachment-focused orientation, although the attachment-focused framework for these therapies is based on many decades of scientific research and study. For example, their selection criteria states that they will only include psychotherapists who practice the Gottman method for couples therapy, and will not include any therapists who practice Emotionally Focused Therapy, even if these therapists are also skilled and trained in therapies in their “evidence-based” list. Such policies seem to be politically motivated in maintaining and perpetuating the turf-wars between different psychotherapy models and in explicitly excluding skilled and trained therapists who don’t conform to their false narrative.

An excerpt of their “evidence-based” claims is shown below:

"Unfortunately, accessing good care is shockingly difficult. Among therapists in health plan networks, just 10% are accepting new patients and exclusively using evidence-based techniques (treatments clinically proven to work). It’s hard enough finding someone to talk to and even harder finding a great therapist with the right expertise for your needs. It’s no surprise that six out of seven people do not receive effective care.

What makes this outcome particularly sad is that effective treatments exist. Short-term interventions like cognitive behavioral therapy (CBT) have been applied to a wide range of problems and are proven to reduce symptoms of depression, anxiety, stress, pain, and insomnia. With CBT, patients identify specific goals and then learn new skills, new ways of thinking, and new behaviors to improve their lives. CBT has been studied in hundreds of clinical trials and consistently delivers impressive results, usually after only 2-3 months. CBT, among other evidence-based therapies, including Acceptance and Commitment Therapy (ACT) and Dialectical Behavioral Therapy (DBT), and the Gottman Method, can be life-changing." (Ebersman, 2018)

Let's look at how this simplistic and misleading marketing pitch propagates the false narrative, as explicated by Shedler (2015):

  1. "evidence-based techniques (treatments clinically proven to work)" and "CBT has been studied in hundreds of clinical trials and consistently delivers impressive results, usually after only 2-3 months." - Yes, CBT interventions have been shown in RCT studies to be effective (albeit with very small gains) in the short-term. However, as already pointed out earlier, these results do not translate directly to practice in real-life. Using the results of RCT trials alone to state that a therapist who utilizes only CBT interventions (or interventions from other models that have been, or will be added to this coveted "evidence-based" list) will be more effective is erroneous and problematic.

  2. "accessing good care is shockingly difficult" and "It’s no surprise that six out of seven people do not receive effective care" taken in its context implies "if it is not an "evidence-based" manualized treatment, it is not good, i.e., therapies not in the "evidence-based" list are inadequate and ineffective.

  3. The statement "[it is] even harder finding a great therapist with the right expertise for your needs."  stated in its context, equates clinical expertise with simply being skilled in certain “evidence-based” techniques, and worse, using these “evidence-based” techniques exclusively. These conclusions are simply marginalizing quality practitioners with clinical expertise, especially in specialized areas of focus, such as developmental trauma.

  4. Making a claim of "exclusively using evidence-based techniques" is absurd in practice. Everything that is said or happens in therapy has to be considered an intervention or technique (for example, sitting in silence with the client). That would imply "sitting in silence" has to be studied by an RCT study (with different contexts/variables), before an "evidence-based" practitioner can do so in session, according to this narrative.

  5. Few clinicians adhere to a puritan model or school of thought, and certainly do not adhere to a script. Most models employ common techniques (such as identifying false/negative beliefs) and all are based on the underlying principles of a healing therapeutic relationship. A study published in the Journal of Consulting and Psychology reports that only half of the clinicians claiming to use CBT use an approach that even approximates to CBT. It doesn't necessarily make such CBT therapists more or less effective than those who practiced what was used in the RCT studies, but shows the problems of classifying an entire class of therapists as "evidence-based" or not, simply based on the model or label clinicians self-identify with.

  6. The practice of psychology is much more complex than simply following a list of techniques for a particular disorder or symptom list.

  7. A rich array of factors contributes to successful therapy (the patient, the therapist, the setting and the therapeutic relationship, to name a few) and not just the treatment method (Marzillier, 2014).

  8. Diagnosis-specific interventions and ESTs are not equivalent to true EBPP. Critchfield and Knox (2010) discuss the conceptual skills necessary for the practice of EBPP, and how therapists who are trained to simply implement ESTs have a relatively rigid skill set of limited applicability.

RCTs do have their place and value in psychotherapy research, especially in the study and fine-tuning of protocols. Manualized treatments and structured protocols are valuable when adapted to the specific patient and circumstances in practice, and CBT and DBT techniques are good skill sets to add to a practitioner’s toolset. The problems arise, however, from the way these studies are being done, and the sweeping conclusions that are being drawn from the research (Marzillier, 2014). Claims made by outcome studies across the spectrum of different psychotherapy models (including person-centered) are suspect. Empirical research shows that most of these therapies are weak treatments. Their benefits are small, few patients get well, and even the small benefits do not last. Troubling research practices paint a misleading picture of the actual benefits of therapies, including systematic bias in research trials, sham control groups, cherry picked patient samples, and suppression of negative findings (Shedler, 2015). Unfortunately, the APA itself is at fault, for providing rigid guidelines on specific effective treatments for different diagnoses, based on these research studies, as though the evidence is strong and conclusive (Marzillier, 2014).

Stuck In The World Of Alternative Facts

The false narrative of "evidence-based" therapies has now been ingrained as fact into influential non-practitioners of psychotherapy (especially medical practitioners, managed care and business leaders), having been repeated for about a couple of decades. Furthermore, since the term "evidence-based" has been appropriated, it becomes difficult to point out the deficits of any of these treatments that have now been labeled "evidence-based" without being accused of being against science (Shedler, 2015). One of the main gains of having this "evidence-based" label is that it magically makes the therapy model "legitimate" having been "scientifically proven" providing easier access to funding for research and practice from managed care and healthcare companies.

Some practitioners, unable to get through to the proponents of these "evidence-based" practices to correct their false narratives, decided that the best way to survive is to join them. So every other model (e.g., DBT, ACT, EMDR) that wants the coveted "evidence-based" status has joined in the madness of continuing these meaningless RCT studies and therefore, knowingly, or unwittingly keep the myth of the false narrative alive.

The result of this "race of madness" is that rather than trying to measure the effectiveness of psychotherapy actually being practiced, we are being asked to practice psychotherapy that can be more easily replicated in RCT studies, to be conferred the valuable title of "evidence-based" therapy. This results in a net reduction in depth and quality of psychotherapy practice rather than an increase in fidelity to effective psychotherapy intervention (Edwards, 2018), and ultimately, is not in the best interests of the consumers of psychotherapy services.

Practitioners of these "evidence-based" psychotherapy models and others who have  something to gain by the false narrative that has taken hold, may decide to disregard all evidence to the contrary, including their own practice-based knowledge, for not wanting to forgo their short-term gains. Their predicament is well described below.

“They participate because they need something—like protection, power, or security. But to get what they need, they must disregard what they see. And to be cliché about it, their short-term gain leads to long-term pain.” (Godwin, 2018, Aug). 

To make any systemic change, those that the broken system currently privileges and rewards (at least in the short term) need to wake up, and join the movement for a call to change the current dysfunctional system. The first step would be to refrain from continuing the false narratives. The next step would be to call out the discrepancies when someone else makes misleading and false claims.

Polarization into two camps

The artificial and unhealthy division of practitioners into two camps (with some open hostility between them), which we can loosely call the "person-centered" aligned with the psychodynamic orientation vs. "techniques-centered" camps has made it difficult to point out the deficiencies in our current “evidence-based” narratives. Criticizing the appropriation of the label "evidence-based" is often wrongly equated to:

  1. Opposing true Evidence Based Practice,

  2. "Being against" the models that have been branded "evidence-based", or

  3. Not wanting to be held to a specific standard when it comes to performance, outcomes and expectations,

which then becomes part of the false narrative. Those that identify with the “techniques-centered” camp are not being manipulative in supporting and propagating the myths of “evidence-based” therapies, but have simply not questioned the truths of the claims, so far, since it matches their own expectations of what makes therapy effective. We all have our prejudices and biases and it is easy to adopt a stance that reinforces them.

The division into being techniques-centered or being person-centered is a false dichotomy. Technique and skill development is necessary, as well as developing the personal and interpersonal qualities of a clinician - sometimes this truth is lost or forgotten in the heat of the either-or argument of tribalism (Godwin, 2018, Sept). However, a model of psychotherapy that sees techniques as just as one of many influences on outcome is better than one that privileges techniques, given that techniques have relatively less influence than the therapeutic relationship (Marzillier, 2014).

Disconnect between practitioners and researchers

Policy makers and influencers who are making these decisions and conclusions that support ESTs (as “evidence-based” treatments) exclusively, are doing so without the input of practitioners and researchers in the field of psychotherapy. Some are being manipulated or misguided into making these misleading claims. Researchers who have published such claims are perhaps too removed from the actual practice of psychotherapy. Most practicing psychotherapists are not trained adequately or are not inclined to critically analyze or question the claims being made by researchers. So perhaps that is why such obvious flaws in the underlying claims of “evidence-based” models in psychotherapy has been allowed to continue for decades.

CONCLUSION

Let us:

  1. Say "NO" to the current "race of madness" in marking a brand X of psychotherapy as "evidence-based" simply by relying on RCT outcome studies. ESTs are not equivalent to EBPP.

  2. Work towards making progress towards actual Evidence-Based practice in psychotherapy. Researchers and practitioners already have a wealth of knowledge, experience and innovative ideas on how to achieve this, especially with the recent advances of technology, which makes it possible to analyze and leverage large amounts of data (quantitative as well as qualitative) using different research methods and utilizing feedback-informed treatments, moving away from the traditional RCT based model, to investigate beyond our traditional symptom reduction outcomes to include person-centered outcomes as well.

PLEASE PROVIDE YOUR FEEDBACK IN THE COMMENTS SECTION BELOW AND LINK OR SHARE THIS ARTICLE TO REACH A WIDER AUDIENCE. THANK YOU!

Acknowledgements

My heartfelt appreciation to the collective wisdom of colleagues, researchers and practitioners in the field of psychotherapy that shaped this article through LinkedIn posts and comments on the initial draft. I was also motivated by John Marzillier’s responses to my queries that helped guide me to the right research references. I thank Jonathan Shedler for taking the time to read my initial draft in spite of his busy schedule. Vincenzo Sinisi provided me with valuable feedback and helped in sharing this article with his extensive network. Finally, I would like to acknowledge Daryl Mahon for his specific comments highlighting the difference between ESTs and EBPP that provided the necessary thread to weave a convincing and clear case for tackling managed care’s falsely restrictive rules on what practitioners may or may not do.

References

  1. APA Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, 271–285. DOI: 10.1037/0003-066X.61.4.271.

  2. Critchfield, K. L., & Knox, S. (2010). Conceptual skills needed for evidence-based practice of psychotherapy. Psychotherapy Bulletin.

  3. Ebersman, D. New funding, same mission to transform mental health. (2018, May 7) Retrieved September 10, 2018 from https://www.lyrahealth.com/blog/series-b-funding/

  4. Edwards, B. G. (2018). The Empathor's New Clothes: When Person-Centered Practices and Evidence-Based Claims Collide in M. Bazzanno (Ed.) Re-visioning Person-Centred Therapy, 1st edition; Routledge.

  5. Godwin (2018, Aug). People Problems. (2018, August 31). https://peopleproblems.org/august-31-2018/

  6. Godwin (2018, Sept). People Problems. (2018, September 7). https://peopleproblems.org/2018/09/

  7. Marzillier, J. (2004). The myth of evidence-based psychotherapy. The Psychologist, 17, [392-395].

  8. Marzillier, J. (2014). Ch.4. The evaluation of trauma therapies in The Trauma Therapies, New York; Oxford University Press.

  9. Norcross, J. C., & Lambert, M. J (2011). Ch.21. Evidence-Based Therapy Relationships, in J. C. Norcross (Ed.) Psychotherapy relationships that work, Evidence-Based Responsiveness, 2nd Edition, New York; Oxford University Press pp 3-21.

  10. Shedler, J. (2015). Where is the evidence for “evidence-based” therapy?. Journal of Psychological Therapies in Primary Care. 4:47–59.

ABOUT THE AUTHOR

pleasanton_therapist.jpg

Vinodha Joly, LMFT is a psychotherapist with a private practice in Pleasanton, California. She specializes in working with adult survivors of childhood trauma, childhood emotional neglect and domestic violence.  Before transitioning to her current vocation as a psychotherapist, Joly worked in research and software development at Silicon Valley high-tech companies, and holds 14 patents in the field of computing. She received her Masters degree in Counseling Psychology from Santa Clara University, California and her Masters degree in Computer Engineering from the University of Michigan, Ann Arbor. Contact: vinodha@vinodhatherapy.com