The Integrator Blog
Share |
Contact Me, Experience, Mission, Sabbatical in Central America, plus
Editorial Advisory Board
Michael Levin
Taylor Walsh
background resources in PDF
Insurance, Integrative Clinics, Industry Summit Reports, News Files '99-'04
some organization links
Professions, Academia, Research, Policy
some CAM/IM publication links
Electronic, Peer-Reviewed, Blogs, More
Bradly Jacobs, MD, MPH, Revolution Health Blog
supported conference
Institute for Health & Productivity Management - Integrative/Complementary Healthcare
Taylor Walsh: Stakeholders Focus on Effectiveness Research at the Institute for Integrative Health PDF Print E-mail
Written by John Weeks   

Taylor Walsh: Institute for Integrative Health Symposium Explores Effectiveness Research for CAM/IM

Summary: This is an auspicious moment for evaluating integrative practices. The Obama-empowered comparative effectiveness research (CER) movement focuses attention on practical evaluations “to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.” Meantime, the new NCCAM strategic plan, if it follows Congress' mandate, will march to a similar drummer. Into this vortex steps the Institute for Integrative Health and the Center for Medical Technology Policy, led by leading NCCAM-funded researcher Brian Berman, MD and former CMWS medical director Sean Tunis, MD, respectively, to convene top scientists in CAM/IM, CER researchers and government agencies to explore the effectiveness ground. Berman calls the gathering the beginning of a process "to make evidence much more relevant, and getting to the essence of the clinical encounter.” The group will create an "Effectiveness Guidance Document" to aid IM/CAM researchers in CER study design. Taylor Walsh, a participant, files this report on the November 9, 2009 symposium. 
Send your comments to
for inclusion in a future Integrator.

Symposium co-sponsor
The research controversy surrounding
the violation of whole-person focused integrative care by the male dominatrix acute/pharmaceutical/RCT research model is so ancient and stuck as to make one nauseated. The continuing failure of research scientists and funders to prioritize evaluation of useful outcomes, however challenging, while sticking to their old patterns of business, however inappropriate, is a kind of water-boarding for the healthcare enterprise of multi-factorial approaches. How long to we have to hold our breath? How do we break this grip?

Happily, we seem to be approaching a moment in which the hold of reductive models may be loosening. Comparative effectiveness research (CER) and observational models are being much discussed in the conventional research community. Obama's dangling of $800-billion for CER is showing a remarkable capacity to inspire new career directions and new questions among researchers. Journal editors, as noted below, are "getting turned on about it."

Symposium co-sponsor
At the same time, the NIH National Center for Complementary and Alternative Medicine is exploring how it will prioritize spending some $600-million over the next 5 years. Presciently it seems, Congress mandated in 1998 that NCCAM focus on effectiveness, outcomes and the kinds of comparisons that will assist stakeholders in decisions about how to integrate CAM modalities and disciplines into conventional delivery systems.  

Into the richness of this potential steps the
Institute for Integrative Health, co-founded by a group that includes NCCAM's best-funded researcher, Brian Berman, MD, and a former NCCAM deputy director, Margaret Chesney, PhD. They worked with Sean Tunis, MD, MSc, a former chief medical officer at Center for Medicare and Medicaid Services and now head of the Center for Medical Technology Policy. Together, they created a top-level dialogue on November 9, 2009 to help, in Berman's words, "to make evidence much more relevant, and getting to the essence of the clinical encounter.”

Taylor Walsh, a new media entrepreneur and journalist in the Beltway
who is also a sometimes Integrator commentarist, was invited to attend as a consumer representative. Walsh last contributed to Integrator content with information about the ARRA grants from NCCAM and, earlier, the priorities of 2005-2007 NCCAM grants. Walsh filed this report.

The Effectiveness Paradigm for Complementary and Integrative Medicine Research: Report from the IIH/CMTP Symposium

- Taylor Walsh

Taylor Walsh: Attendee and reporter
The ”Complementary and Integrative Medicine (CIM) Stakeholder Symposium” held in Baltimore Nov. 9, 2009 created an unprecedented collaboration that included leading integrative medicine researchers, senior members of the comparative effectiveness research community and members of US health agencies.

The symposium was co-organized and hosted by Brian Berman MD of The Institute for Integrative Health and Sean Tunis, MD, MSc, a former Chief Medical Officer at Center for Medicare and Medicaid Services and now head of the Center for Medical Technology Policy.

Berman explained that the purpose of the meeting was to kick-start a process that could take advantage of the changing perceptions of and expectations for how comparative effectiveness research (CER) will be conducted as a result of the new US federal CER initiative created under the American Reinvestment and Recovery Act (ARRA). ARRA will direct some $800 million into CER over the next two years.

Tunis, who led the track on “Economics and Policy” at last February’s “Summit on Integrative Medicine and Health of the Public,” repeatedly encouraged the participants to be aggressive in asserting the value of integrative practice and defining the appropriate methodologies to do so.

“A fresh start is underway with CER,” he told the group.  “The current structure doesn’t work.”

Berman: IHH co-founder and top NCCAM-funded researcher
Berman and Tunis led a tightly structured series of presentations and breakout sessions that included IM researchers from the US, Canada, England and France that included David Eisenberg, MD, Christine Goertz, DC, PhD, Ian Coulter, PhD, Dan Cherkin, PhD, Lawrence Green and George Lewith,
MA, MRCGP, MRCP of Southampton University in England.

Lewith was a principal in a project sponsored by the King’s Fund in 2008 entitled “Assessing Complementary Practice: Building Consensus on Appropriate Research Methods,” that thoroughly examined the factors required to develop and apply “a robust evidence base for complementary practice.”

Tunis: Founder of CMRT, former CMS medical director
US agency participants included Louis Jacques, Director of Items and Devices at CMS; Richard Nahin, PhD, MPH acting director of the Division of Extramural Research at NCCAM; Grant Huang of the Cooperative Studies Program at the VA; and Margaret Coopey of the Center for Outcomes and Evidence at AHRQ.

Experienced members from health services companies long involved in CER, with no previous connection with integrative medicine (IM), presented an overview of the state of the field. The pressure to expand the research paradigm has been building over the last decade and has reached a serious inflection point that could also give integrative practice new evidentiary credibility.

For the IM community, Berman set out simple objectives to start the process.

  • To explore the evidentiary framework for CER in integrative medicine
  • Formulate a strategic framework for efficiently gathering the evidence

The symposium was held in the wake of the creation of the structure and program descriptions for the new ARRA-funded CER initiative.  Much of that work was accomplished by the “Committee on Comparative Effectiveness Research Prioritization” that was established by the Institute of Medicine last spring.

Hal Sox, MD, editor emeritus of the Annals of Internal Medicine and co-chair of the committee, and Tunis, also a member, reviewed the objectives and principles of the CER program in general and its expanding references to evidence in particular.  The objective for the CER program could not be simpler: “To assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.”

Sox said the committee went through 2,600 proposed research topics in order to prioritize a Top 100 by June 30.  Three of those fall within integrative and CAM approaches:

  • Compare the effectiveness of mindfulness based interventions (e.g. yoga, meditation, deep breathing) and usual care in treating anxiety and depression, pain, CVD risk factors and chronic illnesses
  • Compare the effectiveness of acupuncture for various indications using a cluster RCT
  • Compare the effectiveness of dietary supplements (nutriceuticals) and usual care in the treatment of selected high-prevalence conditions. 

Janet Kahn, PhD of the Integrated Healthcare Policy Consortium, who has been deeply involved in ensuring a place for integrative practice in the health reform legislation, asked how these topics were selected.  She noticed that in the IOM’s own summary of 29 primary research areas musculoskeletal topics ranked 10th.  Tunis replied that the topics selected were a function of the people who were members of the committee, and added: “High priority research questions are up to the community.”

“Up to the community” was clearly the theme for the day.

The presentations and discussions described a broadening context for defining evidence through CER, which was met with great interest, some skepticism and realistic cautions from the experienced IM researchers in the room.

An example of a potentially fundamental change: of the 100 research topics selected by the IOM’s prioritization committee, just 49 are random control trials (RCTs).  In its program description the committee wrote that while, “RCTs were the most appropriate methodology for 49 of the topics, (we) also recognized the value of prospective observational studies, database reviews, and systematic literature reviews for studying many research topics.”

And among its specific recommendation cited by Tunis: “The CER Program should devote sufficient resources to research and innovation in the methods of CER, including the development of methodological guidance for CER study design such as the appropriate use of observational data and more informative, practice and efficient clinical trials.”

From this framework Tunis made the case for the use of “Pragmatic/Practical Clinical Trials,” which he said, “seek to inform choices between feasible alternatives; by estimating real world outcome probabilities.”

Tunis was advocating practical trials in 2003 when he was a senior medical officer at CMS, and co-authored an article on the subject for JAMA: “Practical Clinical Trials: Increasing the Value of Clinical Research for Decision Making in Clinical and Health Policy.”  Its summary page included:
“The widespread gaps in evidence-based knowledge suggest that systematic flaws exist in the production of scientific evidence, in part because there is no consistent effort to conduct clinical trials design to meet the needs of decision makers.”
The real-world backdrop for the changing CER landscape was encompassed in presentations from Bryan Luce, SVP of Health Policy for United BioSource Corp., which serves the pharma and health products industries in the US and Europe, and Richard Gliklich CEO of Outcome, which specializes in creating patient registries, a potential research structure for IM/CAM CER studies.

(Outcome is one of 13 AHRQ “DEcIDE Research Centers” (Developing Evidence to Inform Decisions about Effectiveness), a component of AHRQ's Effective Health Care Program, which was created in 2003 “to help clinicians and patients determine which drugs and medical treatments work best for certain health conditions.” Under contract to AHRQ, Outcome produced “Registries for Evaluating Patient Outcomes: A User’s Guide,” a handbook for developing, operating and evaluating the quality of patient registries.

These presentations basically described a traditional CER process that has fallen out of synch with the needs of people who have to make decisions about applying new therapies and products.

Luce was a co-author with Tunis and others of the article: “Rethinking Randomized Clinical Trials for Comparative Effectiveness Research: The Need for Transformational Change,” published online at the Annals of Internal Medicine last June.

His presentation set out limitations of traditional RCTs that, “are too costly, take a long time, don’t provide generalization or deal well with probability, and have a ‘learn and confirm’ aspect that seeks to add to existing evidence.”

As a result he said, payers and practitioners are unable “to predict how new healthcare products and services fit into local clinical settings and patient groups.  CER trials are unsustainable without transformational change in the operations and methods for generating evidence for informed health care decision making.”

The more responsive approach, Luce said, would be to apply “adaptive processes” in trials based on Bayesian statistical approaches that describe results in terms of probability, which he maintained best suits the needs of decision makers.

“The (research) problem often isn’t yes or no,” he said, “but for whom, when, under what conditions, thus requiring flexibility in evidence development.”

Luce presented his “Ideal CER Trial/Learning Process:"

  • Builds on what is known
  • Asks a question the decision makers want answered
  • Chooses a relevant competitor
  • Addresses patient heterogeneity
  • Leans (and adapts) as it accumulates knowledge
  • Stops when “just enough” evidence is generated for informed decision

Such an adaptive framework, Luce maintained, treats CER “more as a process,” rather than the bracketed “start-finish” model of RCTs.  “Adaptive CER trials more closely mimic real-life human learning and decision making,” he said.  With Bayesian analysis this approach would  “produce predictive probabilities of trial success…and predict individual patient results of different treatments.”

After absorbing this vision for what sounded like an Extreme Makeover of the RCT-dominant trial paradigm, participants broke into four groups, each focused on a specific clinical problem:

  • back pain
  • arthritis
  • acute pain
  • other chronic pain

I joined the Back Pain group, where David Eisenberg, MD posed the basic question: whether to approach the issue through a mono-therapy or multi-approach.  Naturally picking the second, the group quickly described familiar barriers and limitations from their own experience.

Ian Coulter, who has been deeply involved in research on the evidence base for CAM funded by AHRQ and NCCAM through UCLA and RAND noted: “The cost effectiveness for CAM is problematic.  The research methodology to do this is not good.”

Eisenberg added: “There are few if any studies on whole systems approach.  Many (potential CER) studies will fail,” an outcome that he suggested would further damage the credibility of all whole systems care.

Several people focused on the patient’s role in trials as a potentially distinguishing factor, particularly if the kind of practical and adaptive trials described by Tunis and Luce were actually considered by CER decision makers.  But as Christine Goertz noted, “We haven’t spent enough time assessing patient perception of outcomes.”

Sitting in our circle was Grant Huang, deputy directory of the Cooperative Studies Program at the Veterans Administration.  Eisenberg asked how the VA might respond to these approaches.  Huang’s advice: “Work with the funders,” he said, to choose the therapy and determine what is important.  “The review committee,” he said, “wants a well designed study.”

Commentary from other participants ran along similar lines.

Jon Tilbert of the Mayo Clinic noted that for patients choosing CAM therapies when they are “at the end of their rope, want to know is it safe?  Is it covered?  Is it worth trying?  Safety trumps efficacy.”

Robert Rothstein, MD, chair of the department of Emergency Medicine at Suburban Hospital in Bethesda, MD, and medical director of its integrative medicine program said, “As a clinician working with MDs, changing physician practice is tough.”

Dan Cherkin, PhD of Group Health concurred: “Given assaults on clinician time. They won’t change practice.  We don’t have the system to encourage change.  It is a huge system issue.”

The firmest cautionary note was made by Eisenberg, who said, “It will be expensive to apply optimal approaches to chronic disease,” he said.  “Who will pay?  The majority of these will fail.  NIH will not.  AHRQ has money, but...  DOD, VA has a toe in the water.  For multi-modal (trials), and repeating, you need ten to thirty projects that show promise.  You’ll need $50 million to run these.”

“Unless Peter Orzag sees the value of this, we’ll be meeting forever,” he said, referring to the White House Office of Management and Budget director.

Tunis responded by recommending an approach in which the community says: “It is our obligation to be ready to say what cost is, what the program is.  Then shop it around.  If it is important, it will have public support.”

Berman concluded the day by describing the symposium as the beginning of a process “to make evidence much more relevant, and getting to the essence of the clinical encounter.”

TIIH and CMTP will develop a draft white paper that will serve to continue the symposium dialogue and reach what Tunis called “a state of shared principles.” Further, an Effectiveness Guidance Document will be drafted, to aid IM/CAM researchers in CER study design.

Reading this on the heels of a week of analysis and writing on what the US Congress wanted from NCCAM when they created the Center in 1998 I was struck by Tunis' description of what CER is supposed to do:
“To assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.”
Anyone who reads NCCAM's Congressional mandate with a beginner's mind will discover a remarkable alignment. Both charges are real world and practical, set to meet present problems, and a little icky for your typical efficacy researcher. Never-the-less, these are the federal mandates.

Beverly Pierce, with IHH affiliate SpeakHealth
For those of you interested, a 35 minute, post-conference conversation among 8 of the participants is posted here. I encountered some great nuggets. One example: Fox, the Annals editor emeritus, spoke of how journal editors are realizing "we're going to eventually much more have to rely on observational research." At a recent conference of other journal editors he "emphasized that we need to get our arms around" these approaches. He said "the journal editors got turned on about it."

IIH, in cooperation with the Imaging Research Center at the University of Maryland Baltimore Campus has also established a web-base at to further the dialogue on this and other themes.
Former hospital-based integrative care leader Beverly Pierce is senior program manager for SpeakHealth. Pierce describes the site this way: "Something that's unique and so interesting about SpeakHealth is that it's completely collaborative. We're not all health care or integrative medicine folks, but artists, film makers, social networking experts and 'persons on the street.' It's a kind of research, really. Can a broad, engaging, public conversation on health (not just healthcare reform) raise consciousness and nudge behavior change?" Take a look and join in.

Send your comments to
for inclusion in a future Integrator.

< Prev   Next >
Integrative Practitioner
The Westreich Foundation
voluntary contributions
Support the work!
All Integrator Round-ups
Integrator Top 10 Lists 2006-2015
Issues #140-#142 Oct-Dec 2015
Issues #137-#139 July-Sept 2015
Issues #134-#136 April-June 2015
Issues #131-#133 Jan-March 2015
Issues #127-#130 Sept-Dec 2014
Issues #123-#126 May-Aug 2014
Issues#119-#122 Jan-April 2014
Issues #116-#118 - Oct-Dec 2013
Issues #113-#115 July-Sept 2013
Issues #110-#112 April-June 2013
Issues #108-#109 Jan-March 2013
Issue #105-#107 Oct-Dec 2012
Issues #102-#104 - July-Sept 2012
Issues #99-#101 - April-June 2012
Issues #96-#98-Jan-March 2012
Issues #94-#95 Nov-Dec 2011
Issues #92-#93 Sept-Oct 2011
Issues #90 and #91 - July-Aug 2011
Issues #88 and #89 - May-June 2011
Issues #86 and #87 - March-April 2011
Issues #84 and #85 - Jan-Feb 2011
Issues #82 and #83 - Nov-Dec 2010
Issues #80 & #81 - Sept Oct 2010
Issues #78 & #79 - July August 2010
Issues #76 & #77 - May June 2010
Issues #74 & #75 - March-April 2010
Issues #73 & #73 - Jan-Feb 2010
Issues #69, #70 & #71 - Nov-Dec 2009
Issues #67 and #68 - Sept-Oct 2009
Issues #65 and #66 - July-August 2009
Issues #63-#64 - May-June 2009
Issues #60-#62 - March-April 2009
Issues #57-#59 - Jan-Feb 2009
Issues #55-#56 - Nov-Dec 2008
Issues #51-#54 - Sept-Oct 2008
Issues #47-#50 - July-August 2008
Issues #46 & -#47 - May-June 2008
Issues #43-#45 Mar-April 2008
Issues #41 & #42 - Feb 2008
Issues #39 & #40 - Dec-Jan '08
Issues #37 & #38 - Nov 2007
Issues #35 & #36 - Oct 2007
Issues #33 & #34 - Sept 2007
Issues #30-#32 - July-Aug 2007
Issues #28 & #29 - June 2007
Issues #26 and #27 - May 2007
Issue #25 - April 2007
Issues # 23 & #24 - March 2007
Issues #21 and #22 - Feb 2007
Issues #19 and & 20 - Jan 2007
Issues #17 and #18 - Dec 2006
Issues #15 and #16 - Nov 2006
Issues #13 and #14 - Oct 2006
Issues #11 and #12- Sept 2006
Issues #9 and #10 - Aug 2006
Issues #7 and #8 - July 2006
Issues #5 and #6 - June 2006
Issues #3 and #4 - May 2006
Issues #1 and #2 - April 2006
All Articles by Subject: 2006
All Articles by Subject: Jan-June 2007
IAYT-Sponsored Series on the Future of Yoga Therapy