February, 1998 Dear Colleague, The American College of Mental Health Administration is proud to have sponsored SANTA FE SUMMIT 1997, the first in a series of consensus development activities to be conducted by the College . We believe that ACMHA is uniquely positioned to provide neutral ground for productive discussion on this and other topics of interest to the mental health and substance abuse fields. The document you hold is the summary report of the work of the SUMMIT and reflects deliberations at the SUMMIT and in ensuing months. We hope that you will comment on and respond to this document, which was created to help focus energy and attention on timely, meaningful measurement of quality in behavioral health services. We found great consensus about the values that must drive the future, and we found much energy for selecting a reasonable number of core indicators. The work ahead is to build on this energy and consensus to tackle the complex methodological issues of measurement, and we trust that this document can help serve as a foundation for the work to come. Please take a moment to review the acknowledgments section at the end of this report. Many, many organizations and individuals have contributed to this work, and the American College is deeply grateful to them. We at the College accept responsibility for the language of the final report; we have tried to accurately reflect the consensus opinions as we understand them. The discussions have been lively and challenging and the issues are daunting; we intend to continue to be actively involved in these exciting discussions. Our starting place in this process was to ensure that changes in the mental health and substance abuse delivery systems remained focused first on the values of good care. We hope that this document will be useful to you and to the field as we realize that vision. Beverly K. Abbott, MSW , President John A. Morris, MSW, Vice-President ACMHA ACMHA and SUMMIT '97 Chair FINAL REPORT OF SANTA FE SUMMIT '97 Preserving Quality And Value In the Managed Care Equation Table of Contents (1) Background and Introduction....................................................pp (2) The Core Set: Values and Indicators endorsed by ACMHA. I. Values...............................................................pp II. Indicators......................................................... pp Outcomes...pp Process...pp. Access...pp. Structure...pp. (3) Special Reports: III. Outcome measures for children and adolescents with serious emotional disorders.................................pp IV. A framework for incorporating prevention............pp (4) Future Development: V. Key measures...................................................pp VI. Issues considered but not recommended ..............pp (5) Acknowledgements and supporting documents: VII. Editor's note.......................................................pp VIII. Acknowledgements and participant list......................pp IX. Appendices and tables ...........................................pp A: Access indicators and proposed measures 1 - 11 B: Process indicators and proposed measures 1 - 8 C: Comparison of performance indicators 1 - 4 ACMHA SANTA FE SUMMIT 1997 BACKGROUND/INTRODUCTION About the College. The American College of Mental Health Administration (ACMHA) was founded in 1979 to advance the field of mental health and substance abuse administration and to promote the continuing education of clinical professionals in the areas of administration and policy. An elected Board of Directors governs the College, and its membership includes leaders in the field from the private and public sectors, a broad range of disciplines, as well as academicians, researchers and consumers. In March of 1996, ACMHA?s Board of Directors voted to build on its diversity to serve as a neutral forum for development of consensus on challenging issues facing the field of mental health and substance abuse treatment and prevention. ACMHA made this commitment real by choosing to host a series of behavioral health summits: collegial efforts to bring together key leaders on targeted subjects. To that end, we will host the SANTA FE SUMMIT ON BEHAVIORAL HEALTH annually at least until the year 2000. SANTA FE SUMMIT 1997. The first SUMMIT was devoted to outcomes/performance measurement, and this document is the result of that initial venture. Leaders from all sectors of the mental health and substance abuse field were invited to attend a working meeting in March, 1997 to see if consensus could be reached on core performance measures for mental health and substance abuse care. Many in the College are concerned that continued proliferation of measures and strategies has the potential to sap both energy and resources from the field. Throughout our work, ACMHA has pushed for simplicity and relevance. In our view, a multitude of technically elegant but uncollectable or ultimately irrelevant measures will fail to move the system forward. Measuring those things that are important to consumers and purchasers, and which can be collected accurately is our goal. This work takes to heart the folk-adage that ?the excellent should not be the enemy of the good.? Approximately 100 individuals accepted the challenge to participate, including representatives of the Washington Business Group on Health, the American Managed Behavioral Healthcare Association (AMBHA), the National Alliance for the Mentally Ill (NAMI), the National Committee on Quality Assurance (NCQA), the Institute of Medicine (IOM), the Substance Abuse and Mental Health Administration (SAMHSA), the Council on the Accreditation of Rehabilitation Facilities (CARF), the National Mental Health Association (NMHA), along with ACMHA members. The College was assisted by an unrestricted educational grant from the Eli Lilly Company to ensure that travel, scholarships for consumers, and other conference needs could be met. Participants in the SUMMIT were divided into five working groups reflecting the domains targeted by those in attendance: PREVENTION, ACCESS (what NCQA refers to as "availability of care"), PROCESS/PERFORMANCE (what NCQA refers to as "appropriateness of care"), OUTCOMES (what NCQA refers to as "outcomes of care"), and STRUCTURE (what NCQA refers to as "systems"). Each workgroup met over the course of two days in Santa Fe, at the end of which each produced a statement of core values for the domain, provisional indicators to capture success in that domain, possible measures and needed next steps. Participants (and in some cases individuals who were unable to be in Santa Fe, but who wanted to be involved in the work) have continued to refine the work begun at the SUMMIT through conference calls, mailings, FAXes, electronic bulletin boards and other mechanisms. The volunteer time commitment alone is an astonishing tribute to the mental health and substance abuse communities. The moderators of the workgroups met in Washington, DC, June 29-30, 1997, to review materials to be included in the first draft of the recommendations of the SUMMIT. Also invited were several primary consumers of mental health services; the College is committed to have the maximum consumer participation possible. This meeting identified several areas that needed additional work, and so ACMHA revised its original time frame for release of the report. The document you see has been through numerous revisions. NEXT STEPS. The College will widely distribute the recommendations that have emerged from the SANTA FE SUMMIT. No document reflects complete consensus among all participants in every area, but we remain interested in seeking the greatest consensus that is feasible. The College has no expectation that this document can supplant the outstanding work of other organizations and groups that have tackled these same issues. We have benefited from the work of , and in some cases the collaboration with, with the Center for Mental Health Services Mental Health Statistics Improvement Project's Consumer Report Card initiative, the Commission on Accreditation of Rehabilitation Facilities (CARF), the National Committee on Quality Assurance (NCQA), the Managed Behavioral Healthcare Association (AMBHA) , the National Association of State Mental Health Director's Research Institute, the National Alliance for the Mentally Ill Round Table, and the Academy for State Health Policy's QISMC project-among others. We do, however, hope that the SANTA FE SUMMIT and these core indicators will help to coalesce some of these diverse efforts, and further the cause of issue refinement and focus to the benefit of all. As this document goes to press, ACMHA is planning a follow-up mini-SUMMIT in April of 1998, to which a group of decision-makers will be invited to review this report and strategize about how to achieve the next steps. Following this meeting, SANTA FE SUMMIT '98 will build on this work on performance measurement and outcomes to look at purchasing value in integrated systems. VALUES. The participants in Santa Fe and the members of the College share a strong investment in a value-driven process, and this may be the chief legacy of the first SUMMIT. Whatever methodological or other weaknesses of this work (and they are inevitable, given the state of the art), there was no debate about the need to anchor outcomes measurement in core service values and the primacy of the consumer of those services. CAVEATS. While we are proud of the leadership provided by ACMHA in this effort, we are cognizant of the complexity of this field of inquiry. Benchmarking, quality improvement, and outcomes measurement in mental health and substance abuse are an emerging area of concentration. Many of the issues with which the SUMMIT participants grappled proved too burdensome to measure, too complex to quantify, or inadequate in other ways. Lest we lose the richness of some of those discussions, we have included a section in this report that addresses issues needing further study and measures reviewed but not chosen. After lengthy debate, we have moved the MEASURES section to a less prominent section of the report, reflecting the lack of consensus and the need for more rigorous validation efforts. We made special efforts to address children and family issues. A subgroup of participants in the SANTA FE SUMMIT have produced a special section on OUTCOMES for children with serious emotional disorders; this work occurred after the Santa Fe meeting. There are references to the special needs of children and their families' needs in each of the major sections of the report, but we acknowledge that the report has not adequately addressed these concerns. In the area of PREVENTION, ACMHA deferred to a work-in-progress that preceded the SUMMIT. The National Mental Health Association, along with the Center for Substance Abuse Prevention, was already in the process of convening a panel of experts and advocates in this area, and so we have merged our process with theirs. The PREVENTION Section is provided to us by that NMHA workgroup, and we are grateful to Rob Gabriele and the NMHA for this significant contribution. YOUR ASSISTANCE. We hope that people from diverse sectors will continue to grapple with the technical and other barriers that hamper progress in assuring quality in mental heath and substance abuse treatment. At the end of this document is a form for comment on the work of the SUMMIT. Please be as detailed as possible in your responses, which should be sent to ACMHA at the address on the form. ABOUT THIS DOCUMENT: The report is presented, in some measure, in priority order. There was greatest consensus about the VALUES, and we believe that this statement of core values is one of the greatest contributions to the field of this work. Somewhat less robust was consensus around core INDICATORS, but we think this is a meaningful set around which to focus debate and further refinement. We feature two special sections: one on CHILDREN'S OUTCOMES and another on PREVENTION. There was least agreement about the MEASURES that are useful to provide the data on the indicators; in some areas we have been prescriptive about measures that seemed most promising, but in other areas we have offered options. Our most controversial elements are those that suggest items pulled from larger, validated instruments. Recommending subsets of existing instruments is a controversial action: there are validity and reliability issues, not to mention copyright and legal liabilities. ACMHA does not recommend use of any instrument or part of an instrument unless the user has addressed the legal and scientific issues. We believe that the mental health and substance abuse field needs to focus its energies on achieving consensus on measures in the near term. USING THIS REPORT: This document is not intended so much as a cook-book as a set of guidelines and principles. We hope that readers and users will help support the driving vision of this work, which is to coalesce energy around a limited number of core indicators and measures-all born out of a vision of quality care. FORMAT: The format of the report is as follows: THE CORE SET: VALUES AND INDICATORS ENDORSED BY ACMHA. Section I: Key value themes. These are the consensus value statements generated in Santa Fe. Section II: Key Indicators. We are using the CARF definition of an indicator: A domain or variable used to point to program quality or performance. SPECIAL FOCUS REPORTS: Section III: Outcomes measures for children with serious emotional disorders. This report was submitted by a sub-group of SUMMIT participants, and then modified based on feedback from a SAMHSA/CMHS funded children's expert panel. Although following the format of the main report, it is in some senses a stand-alone product. Section IV: A Framework for Incorporating Prevention. This is a special submission to ACMHA from the National Mental Health Association's prevention workgroup, and includes selected materials from a larger report to be released by NMHA. FUTURE DEVELOPMENT: Section V: Key Measures. There is a measure or measures for most of the Indicators. We are using the CARF definition of a measure: A specific instrument or data element used to quantify or calibrate an indicator. ACMHA believes that the recommended measures require additional work, and will work with leaders in behavioral health to insure that the refinement occurs. Section VI: Issues considered but not included. These are elements that were generated by the workgroups but not included in the core recommended set. ACKOWLEDGEMENTS and SUPPORTING DOCUMENTS: Acknowledgements and List of Participants: This effort has required the work of many people and organizations. We hope the listing is complete, and apologize for any unintentional omissions. Appendices: The appendices include a variety of supporting documents and tables that were collated or developed for the SANTA FE SUMMIT by HSRI. Editor's Note: Comments from the Program Chair. For additional copies of the report or for other information on SANTA FE SUMMIT or the American College of Mental Health Administration, please contact: Lawrence A. Heller, PhD, Executive Director The American College of Mental Health Administration 7625 West Hutchinson Avenue Pittsburgh, PA 15218-1248 VOICE: 412.244.0670 FAX: 412.244.9916 THE CORE SET OF VALUES AND INDICATORS ENDORSED BY ACMHA Section I: Key Value Themes Section II: Key Indicators SECTION I: KEY VALUE THEMES By the close of the SANTA FE SUMMIT on March 22, the group accepted these value statements as central. They have been edited to make them as clear as possible to a broad cross-section of readers. (1) Consumers and families are at the core of performance measurement. The centrality of the concerns of primary consumers was a consistent theme of the SUMMIT. This value statement is intended to highlight the central role of primary consumers (which includes families of children and adolescents). The role of family members of adults was also addressed, with special attention to their needs for information and involvement to the extent possible. (2) Consumer/customer choice must be a driving value for all systems of care, including their design, delivery, evaluation and accreditation. Choice is an increasingly important element in the delivery of mental health and addictive disorders treatment, and it is a concern that cuts across public and private sectors. In an era of increasingly managed care, choice will sometimes be limited within a plan or system, but this value statement highlights the high priority that should be given to the broadest range of choices possible. (3) Issues of ethnicity, race, age and developmental status, gender, language, culture, spirituality, disability are consciously addressed in ensuring access and availability of services. ACMHA and the SUMMIT participants are concerned that all systems of care become or remain sensitive to their customers. One size does not fit all in providing mental health and substance abuse services, and due diligence is required in the design and delivery of culturally and linguistically competent care. (4) Mental health and substance abuse delivery systems must be accountable to both internal and external stakeholders for meeting the mental health needs of the people they serve in ways that are effective and efficient, and that accountability must be based on reliable, comparable data. To be credible, claims of the quality of care given can no longer rely on assertions of good intent. Data must assist delivery systems to improve, and provide a meaningful yardstick for comparing costs and outcomes to people outside of those systems. . (5) Access to mental health and substance abuse services must be quick, easy and convenient, and outreach and follow-up must be seen as part of the access continuum. Attention to access is not new, but the SUMMIT participants strongly urge continued attention to the basic issues of ease and speed of access. Outreach and follow-up are part of an access continuum. (6) A true public health vision of community health must drive outcomes measurement, which means that universal access and integrated primary and behavioral healthcare are the ultimate goal of effective systems. The participants in Santa Fe recognize the complexity of America?s healthcare systems, but believe that there must be a vision of true community health at the core of all health planning and delivery. This value statement is not intended to denigrate carve-out or specialty care models, but it does reflect the critical importance of coordination. A true public health vision requires attention to prevention and to the health status of entire populations, not just risk-adjusted subpopulations or high-risk populations. (7) Children who have mental health and substance abuse problems: ? should be able to receive effective services in their homes and schools without disruptive removals from either setting; ? should be able to remain safe and out of trouble with law enforcement; ? should remain connected to family and peers while in treatment; ? should receive services that are family focused and health centered. These value statements are difficult to operationalize, of course, but they focus attention on the most normalizing aspects of life for children and adolescents, and away from an over-emphasis on problems and deficits. An orientation towards families as opposed to individuals is not consistent in any area of health care delivery today, and purchasers and providers alike face many challenges in making these value statements have meaning. Nonetheless, ACMHA and the SUMMIT participants view the family focus for children and adolescents as not only a desirable, but a necessary and achievable goal. (8) Adults with mental health and substance abuse problems: ? should be able to maintain a stable, comfortable and safe living environment; ? should be able to engage in chosen, productive daily activity; ? should be able to remain safe and out of trouble with law enforcement; ? should receive treatment that is consumer-centered and which maximizes independence and self-care skills; ? should receive services designed to enhance total health and maintain social connections and improved quality of life. The preceding value statements about adults speak to consumers? concerns about self-determination and dignity. In the field, we are coming to grips with consumers? demands for real power in mental health and substance abuse service design, delivery and evaluation. While the concerns of families of adults who have severe and persistent mental illnesses have legitimate (and all-too-often ignored) needs for information and involvement in the care of their loved ones, the ultimate authority must remain with the adult consumer. These statements also reflect a concern that persons who do not have ready access to appropriate care for mental and addictive disorders can too easily fall prey to incarceration, homelessness and other social problems that could be prevented. SECTION II: THE KEY INDICATORS The SANTA FE SUMMIT workgroups generated mountains of paper and logged hundreds of collective hours of conference calls. In order for this report to receive wide distribution, that level of detail is impossible, the following indicators are much abbreviated and every effort has been made to keep the language simple and unambiguous. The indicators are grouped by four of the domains (OUTCOMES, PROCESS, ACCESS, and STRUCTURE). The domain of PREVENTION is addressed in a separate section [Section IV], and there is a special section on outcomes for a risk-adjusted group of children and adolescents [Section III]. ACMHA is using the definitions of indicators and measures developed as part of CARF?s Strategic Outcomes Initiative. An INDICATOR is a? Domain (e.g., effectiveness, efficiency or satisfaction; either process or outcome) or variable used to point to program quality or performance. A MEASURE is a ?Specific instrument or data element used to quantify or calibrate an indicator. A. OUTCOME INDICATORS The outcome indicators are broken into two sections: the first three indicators are appropriate for all populations, both the commercially insured and working populations, as well as persons with serious and persistent mental or addictive disorders. The remaining measures are seen as relevant primarily for individuals with serious disorders. The measures used to assess performance are different for the two populations. ALL ADULTS: O-I-1. Adults [including those with serious and persistent mental or chemical dependency disorders] reside in their own homes or living arrangements of their own choosing. ACMHA believes that the ability to choose and maintain a stable home environment is a useful indicator of the effectiveness of services for people with mental health and substance abuse disorders. Both types of disorders can expose the individual to the risk of homelessness, transience, or serial supervised living environments. This indicator is intended to capture individuals who are hospitalized, in jail for reasons directly related to their mental illnesses, or homebound. For employed populations, the measures can be less rigorous, but ACMHA believes that it should be part of a core set of domains. O-I-2. Adults [including those with serious and persistent mental or chemical dependency disorders] are working. For employed and commercially insured populations, absences from work and missed productivity are important indicators. For persons with serious and persistent mental and addictive disorders, this is increasingly seen as an essential element of recovery models. For some individuals, ?work? may be understood as meaningful daily activities (including job training, volunteer work, etc.), but consumer participants successfully urged the College to keep the language focused on WORK. Again, different measures are appropriate, but the indicator is part of the core set. O-I-3. Adults [including those with serious and persistent mental or chemical dependency disorders] have good physical health and report good mental health [psychological well being]. It is vital that both general health and mental health be assessed in determining the effectiveness of interventions. Behavioral health services for all populations need to be integrated with primary health care to guarantee the best outcomes. There will be different measures for different population groups, but the value of general health is sufficient to place this indicator in the core set for all populations. INDICATORS FOR ADULTS WITH SERIOUS AND PERSISTENT DISORDERS: O-I-4. Adults with serious and persistent mental or chemical dependency disorders report feeling safe. The issue of safety was highlighted in the work of the SUMMIT. Persons with serious disorders can be at unusual risk for victimization, and often report feeling unsafe because of the reduced social and economic status that often accompanies a chronic illness. Consumers also express concerns about being victimized by public institutions and practices that deprive them of free movement and choices. O-I-5. Adults with serious and persistent mental or chemical dependency disorders avoid trouble with the law. Obviously, people with mental and addictive disorders can commit crimes and have criminal responsibility for the consequences of their actions. This indicator seeks to address the dangers of the use of jails and prisons as a de facto alternative to viable community services. O-I-6. Adults with serious and persistent mental or chemical dependency disorders maintain a social support network. The presence of social supports is well documented as enhancing the quality of life for persons with serious and persistent mental and addictive disorders, and these networks can play a pivotal role in relapse prevention and recovery. O-I-7. Adults with serious and persistent mental or addictive disorders are able to manage their daily lives. Consumers frequently report concerns about managing their daily lives effectively, including symptom management and conflict resolution. This indicator can help assess the effectiveness of a system?s most basic interventions. O-I-8. Adults with serious and persistent mental or addictive disorders report a positive quality of life. Quality of life is the ultimate test of any health care intervention, and this indicator fits with the others as part of a comprehensive core set. B. PROCESS INDICATORS The process indicators are designed to reflect on a system?s performance in serving the needs of the individuals it serves. The essential role of the consumer in all aspects of care is highlighted in this section, reflecting emerging practice for both privately and publicly insured populations. P-I-1. Consumers actively participate in decisions concerning their treatment. This is a bedrock performance issue for ACMHA, and reflects the value statements of the SUMMIT. Consumers are essential partners in all aspects of the therapeutic enterprise. In the case of individuals under 18, ?consumer? should be understood to include family members or guardians actively participating in treatment. [See special section on children and adolescents in Section III.) P-I-2. Consumers who receive inpatient services* receive face-to-face follow up care within seven days of discharge. [*?Inpatient services? are defined as ?24-hour, medically supervised services for a primary mental or substance abuse diagnosis.] There is face validity in the field for the importance of follow-up care for individuals whose mental and addictive disorders are so severe as to require intensive and restrictive levels of care. Failure to engage persons in ambulatory follow-up care after discharge from inpatient treatment is a powerful signal that continuity of care is not present. P-I-3. Consumers with mental health and addictive disorders are engaged in treatment. While it is possible that a single treatment or assessment visit is needed, the norm would be that continuing care is expected with a valid mental illness or substance abuse diagnosis. Failure to continue in regular treatment is especially highly correlated with successful outcomes for persons with substance abuse disorders and persistent mental health conditions. P-I-4. Consumers receive adequate information to make informed choices. This indicator is inextricably tied to indicator P-I-1 and indicator P-I-3 ; active participation in treatment can only be achieved when consumers are provided with useful information about those choices. P-I-5. Consumers receive mental health inpatient services in a voluntary, non-coercive manner. Some persons with mental illnesses may require involuntary hospitalization to protect themselves or others from harm. However, a well functioning service delivery system should be able to minimize unplanned, coercive hospital admissions through care management and effective alternative treatment resources. High rates of involuntary hospitalization may indicate inadequacies in ambulatory care services that are less intrusive/restrictive. P-I-6. Consumers are satisfied with the services they receive. Consumers of substance abuse and mental health services (and their families and guardians) are the best resources for determining whether or not systems are meeting their needs and expectations. It is especially important that clients with these disorders receive services that preserve the dignity and respect of the individual and family. P-I-7. The system of care assumes responsibility for continuous and integrated care appropriate to the needs of children and families. [Also see Section IV, special section on children.] Children with mental health and substance abuse problems are likely to be involved with many systems: schools, child welfare agencies, primary care and pediatric specialty care, juvenile justice and others. Children?s disorders often tend to have periodic changes and care can become episode-driven without consistency. Coordination and integration of care is essential across the developmental span. C. ACCESS INDICATORS The Access indicators identified by the SUMMIT reflect an attempt to move to more meaningful indicators of access than counts of phone rings or drop rates--although these have been useful proxies for access in systems that can track these data. The methodological issues in collecting and analyzing these data are considerable. A-I-1. Consumer experiences of treatment (both positive and negative) are assessed on dimensions of appropriateness, timeliness and sensitivity of services delivered. [Also addressed in P-I-6] Customer satisfaction is another bedrock indicator. The methodologies for measuring this variable are numerous, and need to also include measures of dissatisfaction. A-I-2. Service denials, terminations, or refusals are assessed. Denials, terminations or refusals for services (adjusted for benefits included in a service plan) can serve as a barometer of access. [Serious concerns have been raised about this indicator, because of the issue of the clinical appropriateness of some denials, e.g. the denial of a request for a more restrictive level of care than is indicated for a child, a request for a specific medication that is contraindicated medically, etc.] A-I-3. Penetration rates demonstrate benchmarked levels of service delivery to like populations. The attempt here is to insure that services are at expected levels, neither significantly higher nor lower than is the norm. A-I-4. Access to a full range of services is demonstrable. Easy access to a narrow range of services is not genuine access. (As the folk-adage puts it: ?If all you have is a hammer, everything looks like a nail.?) This indicator would encompass referral linkages and other strategies to offer a comprehensive array. This would have to be risk-adjusted for benefit packages that have limited scope. A-I-5. Children and their families receive the appropriate services that they need, when they need them. This indicator and A-I-6 below were developed and proposed to the SUMMIT process by a group of child and adolescent experts convened by SAMHSA in late 1997. This indicator overlaps somewhat with other ACCESS indicators, but the special emphasis on child and family indicators is seen as essential. A-I-6. Children and their families are being assessed for and offered services at appropriate levels. This indicator seeks to highlight the importance of both penetration and proper matching of children and families to needed levels of care. Of special concern are children being under-identified and hence under-served, as well as children being over-served, for example in the instance of over- or inappropriate utilization of out-of-home placements, restrictive settings , etc. D. STRUCTURE INDICATORS The structure indicators lend themselves more to traditional accreditation and survey techniques, as opposed to true ?outcome? indicators, but are included as relevant to the over-all initiative. S-I-1. The organization?s structure is consistent with the delivery of mental and addictive disorder treatment, with effective consumer and professional representation in policy making. In integrated systems, it is important to ensure that the special needs of persons with mental and addictive disorders are addressed by the structure. The involvement of consumers and professionals in policy is essential in all environments, whether private or public, managed or fee for service. S-I-2. Consumer rights are clearly defined and procedures for resolution of complaints and grievances are in place and easy to use. This is an essential element in most current certification protocols. ACMHA is especially concerned that measures reflect the system?s capacities and performance in making complaint and grievance processes genuinely non-threatening and responsive. S-I-3. Staffing levels are appropriate for delivery of the array of services and provide for meeting the diverse needs of the individuals served, including linguistic and cultural competence. The issue of staffing is enormously complex, and is not responsive to a one-size-fits-all mentality. Of special concern here is the cultural/linguistic element of this indicator. S-I-4. Data on clients is secure, available only to those who need to know. Confidentiality remains an especially vexing concern in the mental and addictive disorders field, not least of all because of federal and state laws regulating access to consumer information. S-I-5. There are appropriate linkages to other service systems with which consumers need to interact. Persons with addictive and mental disorders frequently require services from more than one specialty service, and often need services from other social support systems. In the instance of children, adolescents and their families, this is even more of a need. S-I-6. There is continuity of care within the organization and effective integration with external care-giving systems. This indicator is a close corollary to S-I-5, but an indicator of both internal and external continuity of care is important. S-I-7. There is a single, fixed point of responsibility for each client. All populations need this simple structural support. Consumers and families with complex service needs are better served when there is a single reference point. S-I-8. There is a quality assurance system in place to examine adverse clinical events. Systems need to be able to assess their own vulnerability to incidents such as suicide, aggressive acts and other high risk incidents. S-I-9. Consumers and families are educated about their rights, the array of services available to them, and likely outcomes of treatment interventions. ACMHA is concerned here with improving the effectiveness of systems? communications with their service users. Again, this is a measure with high relevance for all populations. SPECIAL FOCUS REPORTS: OUTCOME MEASUREMENT FOR CHILDREN AND ADOLESCENTS & PREVENTION Section III: Outcomes for Children and Adolescents with Serious Emotional Disorders Section IV: A Framework for Including Prevention SECTION III. A SPECIAL SECTION WHICH ADDRESSES OUTCOMES MEASUREMENT FOR CHILDREN AND ADOLSESCENTS WITH SERIOUS EMOTIONAL DISORDERS This special section was developed after the initial SANTA FE SUMMIT in recognition of the special methodological and other considerations necessary to address outcomes measurement in children, adolescents and families. This initial work focuses on the subpopulation of children most in need of mental health and substance abuse services. This work grew out of the OUTCOMES study panel at SANTA FE, and is included as a virtual stand-alone document. This section models the SUMMIT process from beginning to end, starting with VALUES, identifying INDICATORS for this risk-adjusted subpopulation, and then reviewing the relevant MEASURES for this population. In other sections of the report (PROCESS and ACCESS), the recommendations for children and adolescents are incorporated into the main text. AMERICAN COLLEGE OF MENTAL HEALTH ADMINISTRATION (ACMHA) PROPOSED CHILD OUTCOMES Introduction The American College of Mental Health Administration (hereafter referred to as ACMHA) has endeavored to adapt, for children, adolescents, and their families, the values-based methodology for identification of service outcomes developed at the Santa Fe Summit in March of 1997. ACMHA is aware that numerous organizations (e.g., the Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, National Alliance for the Mentally Ill, American Academy of Child and Adolescent Psychiatry, National Committee for Quality Assurance, American Managed Behavioral Healthcare Association, various foundation and corporate collaborators) are developing comprehensive access, performance and outcome standards for mental health and substance abuse services for adults and children. The ACMHA Child Outcomes Workgroup focused on outcomes but did not attempt to specify values, indicators and measures related to child service system access and performance during its short-lived tenure. For nearly two decades, federal, state, and foundation sponsored initiatives and policies have supported the development of a values-based and principle-driven model of service systems for children and their families known as a system of care (Stroul & Friedman, 1994). Efforts to measure access, structure, performance, and outcomes in systems of care continue as this report is written, and efforts of ACMHA and other groups to develop core performance measures for mental health and substance abuse care for children and families should incorporate aspects of system structure and coordination shown to improve access to an appropriate range of least restrictive services in such systems of care. ACMHA gratefully acknowledges the feedback provided by the individuals convened by the Substance Abuse and Mental Health Services Administration (SAMSHA) to review a working draft of this document. Many of these individuals are participants in one of the afore-mentioned group efforts to develop comprehensive performance and outcomes standards for child and family mental health and substance abuse services. Some of their recommendations were incorporated into the attached document; others were not. Rationale for non-inclusion generally revolved around the extent to which recommendations represented: (1) significant deviations from the values agreed upon at the Summit; (2) the concern of a single advocacy, administrative, or academic group (thus requiring deviation from the consensus process forged at the Summit); and/or (3) significant increases in data collection and response burden. The most vexing challenge faced by the Child Outcomes Work Group was that of reconciling measurement-related concerns with ACMHA's commitment to brevity, pragmatism "simplicity and relevance" -- attributes likely to be valued in the marketplace, where the burden and costs associated with the collection of outcome data will be borne by providers and consumers. Measurement in the field of child and adolescent mental health outcomes is still in an early stage. On the other hand, several symptom checklists, global rating scales, diagnostic interviews, and multidimensional functioning measures have been well validated. However, most are lengthy, require extensively trained interviewers, or rely exclusively on clinician judgment. Thus, the group has recommended subscales from instruments with demonstrated validity, despite the violation of psychometric rigor this strategy represents. Without support for further validation work, the ultimate value of outcomes standards will not be realized. In this vein, the "Methodological Standards for Outcome Measures" prepared by the CMHS Adult Outcome Measurement Standards Committee provides an excellent overview of the methodological issues to be addressed to develop outcomes standards that actually reflect treatment-related change (or maintenance of gains) in consumer populations. In the material which follows, we have laid out; 1) working assumptions; 2) values; and, 3) indicators and suggested measures that index these values. Child Outcomes Work Group Assumptions The four assumptions that guided the group's work are enumerated below. 1. The target population is children and adolescents who exhibit symptoms and impairments sufficient to persistently and significantly interfere with functioning across multiple settings (e.g. school, home, and in community settings). These children might be described as having a serious emotional disturbance (SED) and also as a "risk adjusted" population. As such, they are distinguished from the general population of children and adolescents who, at various stages of development, exhibit problem behaviors and experience emotional distress that are transient in nature. 2. Child outcomes will reflect change at the client (child and family) level. This approach contrasts with a report card approach that reflects the status of a managed care entity with respect to certain indicators of an enrolled population at a single point in time. Assessment over time is essential to examining whether treatment delivered under the auspices of any care entity (managed or not), has an impact, and is particularly critical when dealing with children, for whom changes in behavior, stress, and distress vary (often considerably) over the course of normative development even in the absence of treatment. Thus, it is recommended that data pertinent to the indicators be collected at the outset of treatment, during the course of treatment, upon treatment termination, and up to 18 months year following treatment termination. for youth receiving treatment during any calendar year. 3. Data will be collected from multiple informants, including the child's caregiver and child, archival data from public agencies legally mandated to collect such data (e.g., schools, juvenile justice agencies, child protection agencies), and medical records. 4. Each indicator should be supported by some psychometric data, yet brief, thus requiring careful selection of subsets of items or scales rather than full measures. Although some valid and reliable measures of child and adolescent behaviors, symptoms, and functioning exist, and were reviewed by the work group, many of them require significant administration time and training. Research regarding the sensitivity of these measures to the experiences of the target (risk-adjusted) population of youth, and to treatment-related changes within such a population, is also limited. Moreover, there are no valid measures to index some of the values articulated at the Summit as they relate to children. The group selected measures, subscales, and, single-items supported by psychometric data whenever possible, views further psychometric work as essential to the identification of meaningful but pragmatic outcomes measures, and recommends that such work be supported once final consensus about the indicators to be measured is reached. American College of Mental Health Administration Child Outcomes Work Group Consensus Values, Indicators, and Data Sources VALUE 1:: Youth will reside in the homes of their families 1 Indicator: Children and adolescents should have a stable living situation in a home with a family. RECOMMENDED MEASURES: 1. Child's residence and the restrictiveness of the living environment rated in accordance with Robert Hawkins and colleagues' Restrictiveness of Living Environment Scales (ROLES; Hawkins, Almeida, Fabry, & Reitz, 1992). 2. Number of placement changes experienced by the child during treatment and at 6 month intervals following treatment termination, up to 18 months following treatment. 3. Number of days in out-of-home placement during treatment and at 6 month intervals following treatment termination, up to 18 months following treatment. Data Sources * Caregiver reports elicited at the outset and termination of treatment and at 6 month intervals during treatment and up to 18 months following treatment, the reporting window being the month prior to data collection. * Archival data kept by placing agencies for placements occurring during treatment and at 6 month intervals up to 18 months following treatment, the reporting window being the previous 6 months. The ROLES rating scale can be distributed to these agencies, or to the managed care entity collecting the placement data from these agencies, so that restrictiveness of placements can be scored in a standardized manner. VALUE 2: Youth are engaged in productive activity. Indicator: Youth attend and perform in school (including vocational). RECOMMENDED MEASURES: Data regarding the following are collected from caregivers and school records. 1. Number of days absent 2. Incidents of truancy 3. Number of disciplinary incidents 4. Expulsions 5. Pass/Fail within the last year Caregiver reports are solicited at the outset and termination of treatment, and at 6 month intervals following treatment, the previous month being the time frame for reporting. Archival data regarding these items are obtained for the month prior to treatment, and for 6-month intervals up to 18 months after termination. VALUE 3: Youth have good physical and behavioral health. Indicator: Youth maintain or improve health status and improve behavioral health status RECOMMENDED MEASURES: Physical Health: 1. Youth and caregiver response (about youth) on Item #1 of the Children's Health Questionnaire (CHQ; Landgraf and Ware, 1991, 1996) at the outset, during, and upon termination of treatment, and at 6-month intervals up to 18 months following termination of treatment. 2. Youth pregnancy, as reported by youth and/or caregiver elicited at the outset and termination of treatment, and at 6-month intervals up to 18 months following termination of treatment.. Behavioral Health 1. Reports of suicide attempts made to caregiver, provider, or admitting hospital during treatment and at 6 month intervals up to 18 months following termination of treatment. 2. Symptoms related to mood (negative and positive) as described in CHQ Item 6.1. 3. Symptoms related to concentration, activity, eating, sleep, antisocial behaviors, as described in the Adolescent Outcomes Module (ATOM; University of Arkansas for Medical Sciences, 1995). Substance abuse Recommended: Drug Preference and Drug Involvement subscales of the Drug Use Screening Inventory adapted for adolescents (DUSI; Tarter & Hegedus, 1991). Also proposed: Hair analysis to replace urine screens for youth involved in court-ordered substance abuse treatment, as hair analysis is less intrusive and offers more specific findings. VALUE 4: Youth are safe from criminal victimization, abuse, and neglect. Indicator: Youth will not experience victimization, abuse, or neglect. RECOMMENDED MEASURES: 1. Caregiver reports of criminal victimization of the youth prior to, during, and at 6 month intervals up to 18 months following termination of treatment. 2. Child Protective Service reports of abuse or neglect prior, during, and at 6 month intervals up to 18 months following termination of treatment. VALUE 5: Youth are not in trouble with the law. Indicator: Youth in treatment will not be arrested, detained, or incarcerated. RECOMMENDED MEASURES: Data regarding the following are collected from youth and/or their caregivers and from the archives of county or state juvenile justice authorities/courts. 1. Number of arrests 2. Severity of crime coded in accordance with FBI Uniform Crime Reports 3. Number and length of incarcerations 4. Number and length of probation terms For youth/caregivers and archival sources the reporting interval is the month prior to treatment, months during treatment, and month prior to 6 month intervals up to 18 months following termination of treatment. VALUE 6: Youth have social support. Indicator: Youth have prosocial peers and access to support from adults. RECOMMENDED MEASURES: Peers Subscales from the Family, Friends, and Self (FFS) Assessment Scales (Simpson & McBride, 1992) that tap peer involvement, involvement with peers who get into trouble, and parent familiarity with peers. Responses follow a Likert type format ranging from "none" to "all." Youth responses are elicited at the outset, during, and at termination of treatment, and at 6 month intervals up to 18 months following treatment termination. Adult (non parent) support No valid measure of social support for youth from adults outside the family has been identified yet. VALUE 7: Youth perform developmentally appropriate activities of daily living. Indicator: Youth performs developmentally appropriate self-care and life skills. RECOMMENDED MEASURE: No valid measure spanning childhood and adolescences was identified; the Structured Vineland Scale for youth ages 4-5 and 6-12 has good psychometric properties but requires trained administrators and significant administration time. VALUE 8: Youth enjoy a positive quality of life. Indicator: Youth report having a positive quality of life. RECOMMENDED MEASURE: No valid measure of the construct was identified, and downward extensions of adult quality-of-life measures were deemed inappropriate for youth, for whom the nature of the construct is yet to be defined. Thus, a range of constructs potentially related to a child's sense of well-being (e.g., child's self-esteem or self-efficacy, positive family relations), and valid measures of them were considered. Most of these are quite lengthy. Section #7 of the CHQ (Landgraf & Ware, 1991, 1996) is relatively brief, and ,although entitled "Self Esteem," it appears to tap a child's assessment of quality of life at home, school, and with friends and includes omnibus questions about life in general, and is offered as a potential starting point for measurement of the quality of life construct. References Center for Mental Health Services, Adult Outcome Measurement Standards Committee (1997) Methodological Standards for Outcome Measures (Draft). Rockville, MD: Author. Hawkins, R.P., Alameida, M.C., Fabry, B., & Reitz, A.L. (1992). A scale to measure restrictiveness of living environment for troubled children and youth. Hospital and Community Psychiatry, 43, 54-58. Langraf and Ware (1991, 1996). Child Health Questionnare --Child Self Report Form 87 (CHQ-CF87). Authors. Simpson, D.D., & McBride, A.A. (1992). Family, friends, and self (FFS) assessment scales for Mexican American youth. Hispanic Journal of Behavioral Sciences, 14,1212-1216. Stroul, B.A., & Friedman, R. M. (1994). A system of care for children and youth with severe emotional disturbances. Washington, DC: Georgetown University Development Center. Tarter, R.E., & Hegedus, A.M. (1991). The drug use screening inventory. Alcohol Health & Research World, 15, 65 - 75. University of Arkansas for Medical Sciences (1995). Adolescent Treatment Outcomes Module (ATOM). Little Rock: Author. Acknowledgments Members of the ACMHA Child Outcomes Work Group are: Barbara J. Burns, Ph.D., Duke University Robert Cole, Ph.D., Washington Business Group on Health Connie Dellmuth, M.S.W., Washington Business Group on Health Sonja K. Schoenwald, Ph.D., Medical University of South Carolina Sybil Goldman, Georgetown Child Development Center, also provided assistance, and Kimberly Hoagwood, Ph.D., National Institute of Mental Health made excellent recommendations regarding measures of certain indicators and provided copies of measures and psychometric data for them. In the end, however, instrument length and administration training precluded their inclusion -- and that of other well-validated but lengthy instruments -- in the current report. The assistance of the Substance Abuse and Mental Health Administration in general, and of Eric Goplerud, Ph.D., and Dorothy Webman, Ph.D., in particular, in convening a review group and coordinating feedback mechanisms is gratefully acknowledged. SECTION IV. A Framework for Incorporating Prevention. ACMHA is deeply indebted to the National Mental Health Association for their contribution to the SUMMIT and this report. As noted above, the NMHA had already begun a consensus initiative focusing on prevention, and we are pleased to include in this document several sections prepared especially for the SUMMIT report; additional materials will be forthcoming in a larger, independent document published by NMHA on this topic. We thank Robert J. Gabriele, Senior Vice-President of NMHA for his leadership of this effort. I. The Need for Purchasers to Value Prevention and Incorporate Overarching Values in Preventive Health Care into Health Care Policies Purchasers of managed health care services, including all purchasers for commercial businesses and non-profit organizations and all Federal and state government payers, want to keep their employees/citizens healthy and productive. There is a logical continuum of health care for achieving this goal. Treatment and maintenance services are essential for individuals who have early-stage or chronic physical or mental illnesses, but the continuum is not complete without prevention of the initial onset of disorders and problems. Only when high quality services are provided for the entire continuum will there be a reduction in the incidence, prevalence, and overall costs of the disorders. The purchasers of health care services have had the opportunity to affect a great change in the health status of this country. They are doing this by broadening the delivery system and incorporating prevention along side treatment and maintenance. To do this has required several paradigm shifts: from an illness model to a health orientation, from a discrete intervention model to a holistic orientation, and from a focus on individuals to a focus on families and how illness in one member puts others at risk. Until recently, interest in prevention has been focused on physical illnesses and injuries. Preventive interventions in physical health care have been based on scientific evidence, and much has been learned about immunizations to prevent childhood diseases and changes in diet and exercise to prevent cardiovascular problems. Now there is evidence that risks can also be significantly reduced in the mental health and in physical illnesses whose onset is primarily related to behavior. There are empirically validated studies which demonstrate the efficacy, cost-offset, and improved outcomes for a variety of mental health and medical problems through psychosocial interventions. It is now possible-and prudent-to incorporate preventive services for behaviorally related problems into general health and behavioral health systems of care. OVERARCHING VALUES: The over arching values related to prevention that underlie any system of care that will achieve the purchaser's goal of keeping employees/citizens health and productive include the following: The system of care, including outcomes, must be operationalized, defined and evidence-based. There must be aggressive outreach in service delivery. Access barriers to health care must be eliminated. Consumers must be involved in a process of self-management and empowerment with an existing partnership between consumers and purchasers in determining policy and individual services. Health care services should build on consumers' strengths and increase their potential. Children and families should be the highest priority. A risk and resilience model should be used in assessment and service delivery. A prime goal of service should be to foster healthy life development. Health care should be collaboratively linked to other community resources. Risk profiling of important populations is necessary. The issue of what is important may vary among purchasers and consumers. Risk factors vary in different populations, but they should be identified at rates predicted by epidemiological data. Risk populations should be identified as early as possible with screening at key access points in the health delivery system. The interventions that are provided should be appropriate. The interventions should be associated with the risk profiles of the enrolled population and should focus on risks (resiliencies) amenable to change. Wherever possible, the interventions should be based on identified programs which have a strong evidence base. Enrollees with identified high risk factors should be engaged in the interventions, receive the full course of the program, and be encouraged to maintain their behavioral changes. For those who receive the preventive intervention, there should be an enhanced performance outcome. There should be a reduction of risks and reduction of onset of illnesses in the areas targeted and meaningful to purchasers and customers. Provision of these services will affect the management and finances of purchasers. There will be significant resource (dollars and staff) associated with risk profiling, risk reduction, and resiliency promotion. Appropriate staff will need to be hired to provide the services, and they are likely to need additional training. How Should Purchasers Define Prevention? The classical public health definition of prevention includes primary prevention (focused on incidence), secondary prevention (focused on prevalence), and tertiary prevention (focused on disability). This definition originated at a time when the etiologies of illnesses were thought to be more straightforward than what is now known to be true. Very few illnesses have a single causal agent that can be singularly targeted, such as vaccines to prevent polio. The complexity of risk and protective factors in the etiology of many behaviorally related diseases, both in mental health and physical health, led a committee of the Institute of Medicine of the National Academy of Sciences to seek a definition of prevention that would add clarity to this new knowledge. The committee also realized that by using the term prevention for all health interventions it is difficult to really know what the targets and content of the interventions are. The committee formulated a new classification system for interventions for all of mental health. The system is based on an earlier formulation by Gorden (1985) for physical health, and indeed the new system is equally applicable to interventions for physical problems. In the classification system, the word prevention is reserved for those interventions targeted to a population before the initial onset of a problem or disorder. Treatment involves screening for already existing disorders and appropriate standard care, including efforts to avoid relapse. Maintenance involves after-care service. Unlike the classical public health definition where the use of the word primary implies a type of hierarchy, this system values all three components equally and recognizes that all are necessary for a complete system of care. Purchasers who use this classification system, which has gained wide acceptance in the mental health field, will more easily be able to track the targets and content of the full range of interventions it obtains for its employees/citizens. The risk/protective model is relevant for prevention, treatment, and maintenance, but the risk factors and protective factors are frequently for initial onset, relapse, and chronic morbidity. Why Should Purchasers Value Prevention ? The Costs of Not Providing Prevention Are Huge. Of the ten major causes of disability worldwide, five are mental health and substance abuse problems, with major depression being the first one. Eleven percent of all disability world wide is due to major depression. Of the ten major causes of mortality, seven are directly related to individual behavior (McGinnis and Fogey). The effect of mood on an individual's use of tobacco and alcohol is compelling. Those with high depressive symptom levels -- but not yet major depression -- are much less likely to quit smoking. There Is the Potential that Prevention Will Be Cost Effective. There is the possibility that outlays for health care could be reduced or slowed down. There is both cost effectiveness in the short-term and in the long-term. As more and more people are in managed health care settings, even long-term cost benefits might come back to save the purchaser money. The government particularly has a long-term interest in the public's health and is also significantly impacted by cost shifts from one system to the next. For example, with the child Medicaid population, children with substance abuse and mental health problems frequently end up in the child welfare and juvenile justice systems. The real long-term cost savings may be in these areas even more than in behavioral health care itself. Prevention can have short-term cost benefits that show up department by department and sector by sector. It has the potential of reducing utilization rates and secondary consequences/costs. For example, the potential cost savings from behavioral prevention programs is not going to be just in the behavioral health care component but in other parts of the system where there are also enormous costs, such as visits to the primary care doctor for physical complaints and reassurance. The impact of providing behavioral prevention services must be assessed across general health and behavioral health areas. Most of the immediate outcome cost savings that have been documented are short-term in the physical health arena. There is the possibility is that employee productivity for businesses in the commercial marketplace could be increased. The data on depression alone are suggestive of potential significant savings if effective prevention and intervention strategies are used. It may turn out to be the right thing to do for the people being served. The only way that prevention will be incorporated into policies and purchasing contracts is if purchasers attribute to prevention a high value deserving immediacy and support. The only way that effective prevention will occur is if purchasers insist that the highest quality programs based on the best scientific evidence be used. II. A Risk and Evidence Based Framework for Maintaining and Measuring Prevention Services in Managed Care. Maintaining and measuring high quality, effective prevention services in managed care presupposes that a series of underlying decisions have been made with clarity, integrity, and scientific evidence. These decision points can ensure that the goals are clear, that the best available science is used to select the prevention programs, that the programs are delivered to whom they are intended, and that the interveners are knowledgeable about what they are doing. The following framework, presented in a logical series of steps, can lead to the end goal of a managed care company delivering high quality and effective prevention services to its customers. (1) The term "prevention" should be reserved for only those interventions that occur before the initial onset of disorder. Preventive interventions can be of three types: universal, selective, and indicated. Universal preventive interventions are targeted to the general public or a whole population group that has not been identified on the basis of individual risk. Selective preventive interventions are targeted to individuals or a subgroup of the population whose risk of developing a disorder is significantly higher than average. Indicated preventive interventions are targeted to high-risk individuals who are identified as having minimal but detectable signs or symptoms foreshadowing disorder, or biological markers indicating predisposition for disorder, but who do not meet diagnostic levels at the current time. (2) The target disorders or conditions to be prevented and the target populations for the preventive interventions should be identified and selected by the purchasers (i.e., the true payers of the health care plan) in collaboration with the behavioral health care organization. Thus, the decisions will ultimately reflect the values of both the purchasers and service providers. Through the imposition of contract requirements, purchasers will require service vendors to provide quality preventive intervention services for specific risk groups. Collaboration between the behavioral health care providers and the general (physical) health care providers is crucial so that a) the epidemiology of the covered population is known, and b) preventive intervention outcome measures are not limited to only one area of health. Issues to be considered in the selection process for targeted disorders and conditions and targeted populations include: what disorders or conditions cause the highest cost for the purchaser; based on epidemiological data, what disorders or conditions are most prevalent in the health care population and can high risk groups be identified, and what disorders or conditions cause the most mortality and morbidity for the individual enrollee and his or her family. Morbidity can be reflected in productive work days lost, school days missed, quality of life reduced, and in associated costs, including physical as well as behavioral health care costs and costs borne by other service sectors such as social services, education, and justice. Potential cost-offsets of providing preventive services rather than later treatment and maintenance services should be estimated whenever possible. (3) Only disorders or conditions for which there are known malleable risk and protective factors should be targeted. The science regarding risk and protection is large, changes rapidly, and varies by disorder. Sometimes single risk factors can be identified and targeted, but more often it is the accumulation of risk factors, ideally with the weighting of the relative importance of each factor, that will yield the most potential for prevention of later onset of a single or multiple disorders. The use of a risk reduction orientation within managed care will increase the likelihood that the preventive intervention services will be effective. Science-based, authoritative resources, such as those provided by the National Academy of Sciences, the US Preventive Services Task Force, the Cochrane Collaboration, and many federal agencies and professional organizations, should be utilized to identify those disorders or conditions that have an evidence base sufficient to justify mounting preventive interventions. (4) Only conditions for which there are known, science based preventive interventions should be targeted. Authoritative sources should be utilized. In addition to those listed above, meta-analyses and registries of prevention trials should be reviewed. The preventive interventions that are chosen should have a realistic chance of being delivered at a reasonable cost within a health care setting and/or provide a significant cost offset. (5) Individuals and families should be screened for the occurrence of risk fctors that aare associated with the first onset of a disorder or condition. Screenings must be science based and should include biological factors (including genetic history), psychosocial factors, and early signs or symptoms that do not yet meet full criteria for diagnosis. Protective factors that could offset some degree of risk should also be assessed. The screening tools must be as reliable, valid, and efficient as current science permits. The decision regarding whom to screen is critical. The goal is to be efficient but not miss those with high risk profiles. Members of high risk groups could be identified based on a single and apparent risk factor, such as those who have recently lost their jobs or been diagnosed with a serious illness or children whose parents are alcohol dependent (referred to here as Screen A) and then these identified individuals could be screened more extensively for other risk factors and for protective factors (referred to as Screen B). For example, persons recently diagnosed with a fatal illness could be screened for early symptoms of depression, marital conflict, job stability, and coping style. Adolescents who are known to have a substance abusing sibling could be screened for school attendance and performance, mood stability, coping style, and substance use. Risk assessment tools are more readily available for some conditions than for others. For example, there are some current tools ready for use in screening for risks associated with first onset of depression and substance abuse. The availability of such tools should not act as a constraint because if the scientific evidence regarding risk factors is available, the screening tools can be developed. Screening for risks associated with onset of substance abuse and mental health problems introduces critical issues regarding fairness, labeling, and privacy. The key is for an individual to not get penalized for an assessment of high risk but to get rewarded for behavioral changes that lead to positive health outcomes. Risk profiles should receive the same privacy protections that all health records are accorded. The dual levels of screening that are described above will provide point prevalence data regarding risk factors for particular disorders and conditions for specific populations within the identified health care population. They will also provide point prevalence data on those with already existing disorders. (6) All those identified through screening (levels A and B) as being at high risk for developing a particular disorder or condition must be offered the preventive intervention to prevent that condition. If they accept the offer, the preventive service must be provided. The preventive interventions should be implemented fully, following available protocols. Screening for risk factors for onset of a disorder or condition is likely to uncover some individuals who already have the full disorder or condition, such as unipolar depression, substance abuse, or HIV infection. Referral for further assessment and treatment for these individuals is essential. (7) Prevention interveners must be thoroughly trained in the relevant risk assessment tools and in the implementation of each specific preventive intervention that is delivered. Training will help ensure that the programs are delivered to the appropriate high risk groups and that standardized full implementation with fidelity to the original program designs is achieved. Ongoing supervision of front-line interveners is desirable to maintain enthusiasm, ensure fidelity, and decrease personnel turn-over. (8) The following process and capacity measures (sometimes called intermediate goals) should be documented: Percentage of total personnel who have received training in risk assessment and preventive interventions; the total number of prevention interveners; and a description of the type and method of training received. Percentage of total enrollees who are informed regarding risk identification and the provision of preventive services for mental health problems and substance abuse within the health care organization; a description of the type and method of communication, such as newsletters, pamphlets, or discussion with a service provider. Percentage of the health care population screened (levels A and B) with assessment tools for risk factors associated with first onset of targeted disorders or conditions; a description of the tools, methods, and results (including gender, age, health care usage, and risk status). Percentage of the health care population who self-refer for preventive services; a description regarding what services are most frequently requested. Of those who are screened (levels A and B) and determined to be at high risk, the percentage who are offered and referred to preventive services within the health care organization. Percentage of those who utilize the prevention services, including those who receive part of the program and those who receive the full intervention; a description of the preventive services. (9) The intermediate performance measures, also known as proximal outcomes, should focus on risk status within the targeted population. Change or lack of change on the targeted risk factors should be documented for each individual and the risk population as a whole (the latter is the incidence of risk factors). For example, measures could include fewer depressive symptoms in a high risk but non-clinical population who received a behavioral/cognitive preventive intervention, fewer incidents of reported unprotected sex among adolescents who received life skills training, and higher birth weights in infants whose mothers had been home visited as part of prenatal care.) These intermediate performance measures should be collected and recorded with systematic regularity, such as every six months for measures related to depression in the elderly. A consistent reduction in risk over time is likely to lead to a positive distal outcome, also known as a key performance outcome measure. (10) The key performance measures, also known as distal outcomes, should focus on the primary disorder or condition to be prevented. It is these outcomes that are the ultimate targets of the interventions. Such measures could include prevention of premature delivery and low birth weight; prevention of the onset of unipolar depression during an adolescent's high school years; prevention of HIV infection during an individual's college years; prevention of teenage pregnancy; prevention of substance use during a child's middle school years; and prevention of unipolar depression in the year following an individual's first heart attack. All measures must be quantifiable, easily understandable, and valid. [Meaningful, measureable and manageable in the ACMHA taxonomy.] Because most outcomes of prevention services are not absolutely tied to a specific date for onset, an explicit time frame should be part of the outcome measure. For example, birth weight has a specific time for onset, but depression does not. Prevention of depression during an academic school year or during adolescence is a realistic goal whereas prevention of depression for a lifetime is not. For some individuals prevention may be a delay of onset, but this too can save years of suffering and cost. Also, there appear to be critical periods in life development when preventive interventions may be especially potent. For example, preventing major depression during a woman's pregnancy and the following year may have major effects on infant health and development. Comparison should be made between the outcomes obtained in the health care setting and the original research. If possible, incidence and point prevalence data for the targeted disorders and conditions should be gathered on a periodic basis. Collection of additional data regarding distal effects, especially on general health and use of other medical services, is encouraged. These quantitative performance measures should be used to track progress over time toward each specific objective originally identified by the health care purchasers in collaboration with the behavioral health care organizations. (11) Documentation of the costs of risk assessments and prevention programs should be collected not only for the whole serviced population but also on an individual basis. Any future cost-savings analyses will require these figures. The screening for risk factors will yield some positive cases of already existing disorder, and these individuals will be referred to treatment. Therefore, somewhat ironically, screening for prevention can result in higher treatment costs for the health care organization, and these extra costs need to be considered. Summary The use of the above step-by-step framework provides a reasonable assurance that the prevention services that are delivered will be of high quality and will be effective. The framework does presuppose that the health care organization has the capacity to carry out these tasks. A social marketing approach will be needed to convince providers that preventive services and this framework for ensuring quality and effectiveness should be part of their comtracts with health care organizations. Purchasers will need to be convinced that prevention programs: have face validity; have an attractiveness that will appeal to customers; have performance measures that can be quantified; have the potential to pay-off as investments; and have a sufficient universality that they will appeal across public and private sectors. FUTURE DEVELOPMENT ACMHA MEASURES REVIEW & CONSIDERED BUT NOT ADOPTED Section V. ACMHA Measures linked to Indicators Section VI. Items considered but not adopted. SECTION V: KEY MEASURES This section will include the proposed measures for each of the indicators selected for the core set. This was the part of the SUMMIT?s work that proved the most difficult for participants. There is considerable debate in the field about the efficacy of different measurement instruments, the burden of collecting data, data comparability, etc. ACMHA welcomes the work of our colleagues in refining and improving these recommendations. We propose that as the field accepts the VALUES and INDICATORS proposed in this document, that there will be a new consensus to tackle the methodological disputes in these targeted areas. The measures will follow the same order as the indicators. In reviewing our work, HSRI offered some observations about issues that are cross-cutting for measurement: (1) For indicators/measures relying on survey data , what is an appropriate sample size? Consensus needs to be reach about the size of differences that are sought, so that power analyses can be conducted and sample size determined. (2) How should the sample be drawn? Before an indicator requiring a survet could be fully implemented, guidelines would need to be agreed upon regarding the survey administration (i.e., mailed? Phone? in-person?), minimum acceptable completion rates, and the larger sampling frame (i.e., all enrollees? All enrollees with at least one encounter? Etc.). (3) How will risk-adjusting and benchmarking be handled? All measurement instruments will need to have demonstrable validity if they are to have utility across plans, populations and settings. (Dr. Joe Thompson of NCQA highlights the ditinction between measures that are useful for internal quality improvement (QI) versus those that have reliability for quality comparison (QC), the latter having to meet a higher standard of scientific validity. There are tables in the appendix which give further detail. Measures have been selected based upon the judgement that they are measurable, manageable and meaningful. Manageability reflects the relative ease/burden of collecting and analyzing the data collected. Measurability refers to the extent to which a measure can give quantifiable and comparable expression of the domain being studied--the scientific dimension. Meaningfulness refers to the relative utility of the measure to the mental health and substance abuse fields: Can the resulting information be useful to consumers and purchasers in making decisions? Can the information help providers manage better? A: OUTCOME MEASURES INDICATOR ONE. (O-I-1). Adults [including those with serious and persistent mental or chemical dependency disorders] reside in their own homes or living arrangements of their own choosing. PROPOSED MEASURES: O-M-1. Consider categories from the Lehman Quality of Life Inventory (Brief version): (a) What is your current living arrangement, and (b) How much choice did you have in selecting the place where you live. There is a rating scale from "Total" to "None." [Note: Use of these questions without checking for copyright and validity concerns is NOT recommended.] O-I-2. Adults [including those with serious and persistent mental or chemical dependency disorders] are working. RECOMMENDED MEASURE: O-M-2: For the risk-adjusted population of persons with severe and persistent mental illnesses, consider two items: (1) from the Lehman Quality of Life Scale (Brief version): (a) "What kind of work do you do at the present time? "; (2) from the International Association of Pyschosocial Rehabilitation (IAPSRS) Programs "Toolkit for PsychoSocial Rehabilitation Outcomes: (b) "How many hours a week do you usually work?" [Note: Use of these questions without checking for copyright and validity concerns is NOT recommended.] O-I-3. Adults [including those with serious and persistent mental or chemical dependency disorders] have good physical health and report good mental health. RECOMMENDED MEASURE: O-M-3: Consider selected scales from the SF-12 (physical and mental component scales), BASIS 32 (depression and anxiety, psychosis, and impulsive addictive behavior scales), or MHSIP Report Card (symptoms, medications, and side effects questions). [Note: Use of these questions without checking for copyright and validity concerns is NOT recommended.] O-I-4. Adults with serious and persistent mental or chemical dependency disorders report feeling safe. RECOMMENDED MEASURES: Consider modified items from Lehman Quality of Life Inventory: (a) In the past have you been a victim of a violent or non-violent crime?; (b) How safe do you feel where you live? [Note: Use of these questions without checking for copyright and validity concerns is NOT recommended.] O-I-5. Adults with serious and persistent mental or chemical dependency disorders can avoid trouble with the law. RECOMMENDED MEASURE: Consider adaptation from Lehman QOLI: In the past have you been arrested or picked up for any crime? [Note: Use of these questions without checking for copyright and validity concerns is NOT recommended.] O-I-6. Adults with serious and persistent mental or chemical dependency disorders maintain a social support network. RECOMMENDED MEASURE: Consider multiple items from the Lehman QOLI: (a) In the past how often did you: visit with a friend not living with you? Telephone a friend? Make a plan ahead of time to do something with a friend? Spend time with someone like a girlfriend or boyfriend? Talk with a member of your family on telephone? Get together with member of family? [Note: Use of these questions without checking for copyright and validity concerns is NOT recommended.] O-I-7. Adults with serious and persistent mental or addictive disorders are able to manage their daily lives. RECOMMENDED MEASURE: Consider selected items from the MHSIP Consumer Report Card Survey, specifically those relating to: "I deal more effectively with daily problems (Q26); "I am better able to control my life." (Q28); "I do better in my leisure time." (Q34); "I have become more independent." (Q37); "I am more effective in getting what I 'want' [note: word is 'need' in MHSIP questionnaire]. (Q39); "I am better able to deal with crises: (modified from language in Q40: Original language: "I can deal better with people and situations that used to be a problem for me." ) O-I-8. Adults with serious and persistent mental or addictive disorders report a positive quality of life. RECOMMENDED MEASURES: Consider single item from Lehman QOLI:"How do you feel about life in general? "; B: PROCESS/PERFORMANCE MEASURES P-I-1. Consumers actively participate in decisions concerning their treatment. RECOMMENDED MEASURES: Consider two questions from MHSIP consumer survey: (a) I, not the staff, decide my treatment goals (Q19) ; (b) I felt comfortable asking questions about my treatment and medications (Q12). [Note: Use of these questions without checking for copyright and validity concerns is NOT recommended.] P-I-2. Consumers who receive inpatient care* receive face-to-face follow up care within seven days of discharge. [*?Inpatient care? is defined as ?24-hour, medically supervised care for a primary mental or substance abuse diagnosis.] RECOMMENDED MEASURE: The total number of discharges from 24-hour, medically supervised care for a mental health or substance abuse diagnosis that were followed by at least one non-emergency, face-to-face mental health or substance abuse treatment visit within seven days, divided by all discharges from such settings, during a 12 month period. P-I-3. Consumers with mental health and addictive disorders are engaged in treatment. RECOMMENDED MEASURE: The total number of enrollees receiving one and only one mental health or substance abuse service in the past year, divided by the total number of enrollees receiving more than one mental health or substance abuse service in the same year. P-I-4. Consumers receive adequate information to make informed choices. RECOMMENDED MEASURE: Consider three questions from MHSIP survey: (a) I felt comfortable asking questions about my treatment and medication (Q12); (b) I was given information about my rights (modified from Q14); (c) I was told what side effects to watch for (modified from Q17). [Note: Use of these questions without checking for copyright and validity concerns is NOT recommended.] P-I-5. Consumers receive mental health inpatient services in a voluntary, non-coercive manner. RECOMMENDED MEASURE: Total number of admissions to 24-hour, medically supervised residential mental health and substance abuse treatment to which consumers are admitted (committed) involuntarily, divided by the total number of 24 hour, medically supervised admissions in a 12 month period. [Note: ACMHA acknowledges significant problems with data collection of this data because of variability in plan design, methodology for coding encounters by individual versus family, variability in location of services, including primary care settings, etc.] P-I-6. Consumers are satisfied with the services they receive. RECOMMENDED MEASURE: Consider selected questions from the MHSIP Consumer Survey: (a) I like the services I receive from my mental health or substance abuse provider (modified from Q01); (b) I would recommend my mental health or substance abuse provider to a family member or friend (modified from Q03); (c) I feel that I was helped by the services I received; (d) I feel that I was treated with dignity and respect (modified from Q20-n, Q14); (e) I feel that I was free to ask questions about my mental health and substance abuse treatment (modified from Q12); (f) I feel that my provider is sensitive to my cultural and ethnic background (Q20-n). [Note: Use of these questions without checking for copyright and validity concerns is NOT recommended.] P-I-7. The system of care assumes responsibility for continuous and integrated care appropriate to the needs of children and families. [Also see Section IV, special section on children.] RECOMMENDED MEASURE: See special section on children, Section III. ACCESS MEASURES A-I-1. Consumer experiences of treatment (both positive and negative) are assessed on dimensions of appropriateness, timeliness and sensitivity of services delivered. RECOMMENDED MEASURES: (a) Consider customer satisfaction with initial access measure from AMBHA PERMS: Was the amount of time you had to wait for your first appointment , , , . [Note: Use of these questions without checking for copyright and validity concerns is NOT recommended.] (b) Consider selected questions from MHSIP Consumer Survey: "I was unable to get the services I thought I needed" (Q09-n); "I was able to see a psychiatrist when I wanted to" (Q10); "Staff were willing to see me as often as I felt it was necessary" (Q06); "Staff were not sensitive to my cultural/ethnic background" (Q 20-n) ; "The location of services was convenient" (Q05) ; "Services were available at a times that were good for me" (Q08); "I was unable to get some services I wanted because I could not pay for them."(Q04-n). [Note: Use of these questions without checking for copyright and validity concerns is NOT recommended.] A-I-2. Service denials, terminations, or refusals are assessed. RECOMMENDED MEASURES: NO MEASURE IDENTIFIED. There are definitional, data source, and other problems with this indicator and its measurement. Considerable attention is being given to external review of this indicator. A-I-3. Penetration rates demonstrate benchmarked levels of service delivery to like populations. RECOMMENDED MEASURE: AMBHA PERMS 1.0 . A-I-4. Access to a full range of services is demonstrable. RECOMMENDED MEASURES: (a) MHSIP Report card measure of ready availability; requires more than one question; (b) review of contract provisions/external review protocols; (c) utilization rates by service type per administrative data base. A-I-5. Children and families receive the appropriate services the need, when they need them. RECOMMENDED MEASURES: The potential sources for this information include CAHPS, FSQ-R/YSQ-R, MHSIP Report Card, AMBHA PERMS, and YRBS. (a) Self/Family Report/Survey, which includes consideration of culture, geographic spread, clinical use and timeliness; and (b) information through service and administrative data bases retrieved through queries such as: Length of time from first appointment to second appointment by: (1) frequency distribution of % initial contact to first appointment; (2) % persons who show for first appointment within 30 days of initial contact. Length of time from first to second appointment by: (1) frequency distribution of % of initial appointment to second; (2) % of persons appearing for second appointment within 30 days of first. Percent of consumers identified via Geo-mapping to be within 30 mile radius of provider. (c) Information gathered through accreditation processes such as NCQA, CARF or JCAHO. An important variable accessible through this mechanism might be cultural competence/sensitivity as measured by provider offering translation or multi-lingual services if >10% are identified as non-English speaking. A-I-6. Children and the families are being assessed for and offered services at appropriate levels. RECOMMENDED MEASURES: These measures could be gathered through service or administrative data base queries: (a) penetration rate by age, sex, and population for services to clients with mental health and substance abuse primary diagnoses, as benchmarked against epidemiologically based predicted rates. (b) Penetration rate in the primary health care system of clients by age, sex and population as compared to predicted rates; (c) Follow-up and transition data monitoring the intervals of time between providers on referral; #days/referral, benchmarked against mean days for the system. D: STRUCTURE MEASURES As noted in the introduction, the structure measures tend to be the more traditional accreditation measures. In keeping with the ACMHA concept of not duplicating efforts of existing bodies, our recommendation in this area is that systems use those elements of existing accreditation surveys that address these indicators. CARF, NCQA, JCAHO, HCFA and others all have instruments or survey questions to assess these indicators. It is the ACMHA position on these measures, that the standard should be: A benchmarked or nationally accepted measure or survey that adequately supports a finding on the indicator in question. ACMHA believes that no survey system is adequate unless all of the ACMHA indicators are covered. SECTION VI: MEASURES AND INDICATORS CONSIDERED BUT NOT RECOMMENDED AT THIS TIME Performance Indicator: P-I-5. Psychotherapeutic medications are used appropriately. The misuse of psychotherapeutic agents is a source of grave concern for consumers, professionals and purchasers alike. The implications for ineffective (or worse, counter therapeutic or dangerous) treatments are significant. Corresponding Measure: P-I-5. Psychotherapeutic medications are used appropriately. RECOMMENDED MEASURE: None. Discussion: This is an important measure, which has face validity and is seen as an important quality of care indicator. However, an existing/established instrument for this indicator could not be found. Moreover, while the concept of ?appropriateness of pharmacotherapy? is easy to appreciate, there are actually few, if any, established criteria for best practices and there is a wide range of ?acceptable practices. HEDIS 3.0 included a test measure regarding the use of antidepressant medications, but a concern of the workgroup was that this measure was narrow and addressed the issue of unnecessary prescription of medications, not the broader focus of use or lack of use of appropriate medications. The workgroup is committed to the notion of monitoring and evaluation pharmacotherapy practice. Of special interest was the work of Lantz, Giambanco, and Buchalter in a recent issue of Psychiatric Services (47:9, 951-55). The process work group recommends development of a measure based on this work, including a matrix of drug utilization sorted by major diagnostic categories. Other data variables (gender, ethnicity, age) were also proposed. This measure is seen as dependent on the existence of pharmacy data bases and electronic patient record technologies. ACKNOWLEDGEMENTS & SUPPORTING DOCUMENTS Editor's Note As Program Chair of SANTA FE SUMMIT '97, the task of editing this report naturally fell to me. In a very real sense, it has been a joy. Yes, there have been missed deadlines, confusions about intent and language and format, disagreements about what's in and what's out, and the expected share of baffling computer accidents. But in the process I have worked with colleagues across the country who have been unfailingly responsive, cordial, competent and committed to this effort. The hours of professional and personal time that are reflected in these relatively few pages is almost incalculable-and the value of the process alone, without price. Many, many hands have made this work possible, and yet I will take the risk of singling out a few whose contributions were so signal that without them the work would not have been completed. They are: Larry Heller, our ACMHA executive, whose tirelessness and patience are awesome. No request, however trivial, was ever met with anything less than prompt, efficient results. He served as father/confessor and allowed me to air my worries and frustrations-and always brought me back to the potential of the work. My colleagues Eric Goplerud of SAMHSA, Neal Adams of Santa Clara County Mental Health, ACMHA President Bev Abbott of San Mateo County Mental Health, and Sonja Schoenwald of the Medical University of South Carolina-all were faithful and generous with their time, advice, and expertise far beyond what could be reasonably asked of busy people. Betty Downes of Santa Fe, without whose astonishing energy we would not have ended the original SUMMIT meeting with the written product that formed the foundation of this report. Mary Jane England and HG Whittington, whose wisdom and humor and vision helped make this task seem not only doable, but essential. Their personal warmth and support served to make more than one rough patch smooth. Jay Scully, my colleague (and , incidentally, my boss), who appreciated the importance of the work not only for South Carolina, but for the field, and who unfailingly supported my involvement. And finally, of course, my family: Jennie, Dan and Paul have endured my endless stories of successes and setbacks with their characteristic patience and humor. They often even convincingly feigned interest-who could ask for more? They're the best. John Morris Columbia, SC January 28, 1998 VIII. ACKNOWLEDGMENTS The College recognizes with special gratitude the following: Dr. Robert Browne and Mr. Curtis McManus and their colleagues at the ELI LILLY CORPORATION for an unrestricted educational grant that made SANTA FE SUMMIT 1997 possible. Dr. Eric Goplerud and Dr. Ron Manderscheid of SAMHSA for the technical and financial support that enabled the work groups to continue after Santa Fe. Dr. Mary Jane England and the Washington Business Group on Health for her role in the planning and execution of the SUMMIT. Sybil Goldman, MSW and the Georgetown Technical Assistance Center who served as contract agent for the SUMMIT follow-up. Dr. Betty Downes of Santa Fe, New Mexico and Dr. Lawrence Heller of the ACMHA office, without whose technical and strategic assistance none of this work would have been accomplished. Our colleagues at the Human Services Research Institute (HSRI) who provided technical assistance to the workgroups and did much of the work on measurement. Rob Gabriele of the National Mental Health Association, who agreed to assume leadership of the Prevention work for this report. Dr. Dan Fisher and Ms. Darby Penney who met with the ACMHA Board immediately following the SUMMIT to review our work, and to Ms. Laura vanTosh and Ms. Darby Penney for their participation in a working meeting in DC to produce the first comprehensive draft of the SUMMIT report. The Conference Planning Committee and to the Work Group Moderators for their leadership, with special thanks to Dr. H. G. Whittington and Dr. Mary Jane England for their skill in leading the dialogues in Santa Fe. To the following individuals who served as presenters at the SANTA FE SUMMIT (in order of their appearance): Dr. Joe Thompson representing the National Committee on Quality Assurance (NCQA). Dr. John Bartlett and Dr. Clarke Ross representing American Managed Behavioral Healthcare Association (AMBHA). Dr. Bob Cole (of the Washington Business Group on Health) and Ms. Laurie Flynn (of the National Alliance for the Mentally Ill) representing the Foundation for Accountability (FACCT). Dr. Ron Manderscheid representing the CMHS Mental Health statistics Improvement Project Consumer Report Card initiative. J. Alex Valdez, Esq., representing the New Mexico Department of Health. Ms. Laurie Flynn and Dr. Donald Steinwachs representing the National Alliance for the Mentally Ill?s NAMI Roundtable. Dr. Constance Pechura representing the Institute of Medicine. Dr. Bob Browne representing Eli Lilly. Dr. Eric Goplerud representing the Substance Abuse and Mental Health Services Administration?s Managed Care Initiative. Dr. Barbara J. Burns (University of North Carolina), Ms. Barbara Huff (Federation of Families for Children?s Mental Health), Ms. Sybil Goldman (Georgetown Technical Assistance Center), and Dr. Sonja Schoenwald (Medical University of South Carolina) who presented a special panel on children. Finally, it would be impossible for ACMHA to adequately express our gratitude to the scores of people who gave hours of their precious time to participate in follow-up conference calls, read countless drafts of committee reports and summaries, and who showed a remarkable spirit of commitment to this challenging project. Listed below are the members of the six work groups: ACCESS: Dr. Rama Khalsa and Dr. Gail Barton, Moderators MEMBERS: CHILDREN: Dr. Barbara Burns, Ms. Connie Delmuth, Dr. Robert Cole and Dr. Sonja Schoenwald, with assistance from: OUTCOMES: Ms. Beverly Abbott and Dr. Sheila Baler, Moderators MEMBERS: PREVENTION: SANTA FE GROUP: Moderators: Dr. Nancy Valentine and Ms. Sybil Goldman Members: CSAP/NMHA GROUP: Moderator: Rob Gabriele Members: Mr. J.B. Bixler Executive Director Association of Community Mental Health Authorities of Illinois National Association of County Behavioral Health Directors Rural Route 1 Box 269 Springfield, IL 62707 P: (217) 544-5299 F: (217) 544-8092 Mr. Greg Brannan Director of Business Development Charter Behavioral Health System of Maryland at Potomac Ridge 14901 Broschart Road Rockville, MD 20850 P: (301) 251-4576 F: (301) 424-3841 Dr. William Bukoski Associate Director of Prevention Research Coordination National Institute on Drug Abuse Parklawn Building, Suite 9A 53 5600 Fishers Lane Rockville, MD 20857 P: (301) 443-2974 F: (301) 443-2636 Dr. Robert Cole Director Washington Business Group on Health Suite 800 777 North Capitol Street, NE Washington, DC 20002 P: (202) 408-9320 F: (202) 408-9332 Mr. Bruce Emery Director National Technical Assistance Center for State Mental Health Planning Suite 302 66 Canal Center Plaza Alexandria, VA 22314 P: (703) 739-9333, ext. 28 F: (703) 548-9517 Dr. Michael Fishman Assistant Director Division of Maternal, Infant, Child and Adolescent Health Maternal and Child Health Bureau Parklawn Building, Room 18A-30 5600 Fishers Lane Rockville, MD 20857 P: (301) 443-5372 F: (301) 443-1296 Dr. John Gates Director The Carter Center One Copenhill Atlanta, GA 30307 P: (404) 420-5165 F: (404) 420-5158 Ms. Sybil Goldman Director National Technical Assistance Center for Children's Mental Health Georgetown University Child Development Center Suite 401 3307 M Street, NW Washington, DC 20007 P: (202) 687-5052 F: (202) 687-8899 Dr. Eric Goplerud Associate Administrator Office of Manage Care Substance Abuse and Mental Health Services Administration Parklawn Building, Room 13C - 15 5600 Fishers Lane Rockville, MD 20857 P: (301) 443-2817 F: (301) 443-8711 Mr. Thomas Gullotta CEO The Child and Family Agency 255 Hempstead Street New London, CT 06320 P: (860) 443-2896 F: (860) 442-5909 Mr. Pete Holt Government Programs Value Behavioral Health 3110 Fairview Park Drive Falls Church, VA 22042 P: (703) 208-8519 F: (703) 876-5644 Dr. Leighton Huey Professor, Vice Chairman, and Medical Director Department of Psychiatry Dartmouth Medical School Dartmouth-Hitchcock Medical Center One Medical Center Drive Lebanon, NH 03756-0001 P: (603) 650-8558 F: (603) 650-5842 Dr. Nancy Kennedy Director Office of Managed Care Center for Substance Abuse Prevention Substance Abuse and Mental Health Services Administration Rockwall II, Suite 901 5600 Fishers Lane Rockville, MD 20854 P: (301) 594-0788 F: (301) 443-1548 Ms. Amy Lockhart Presidential Management Intern Office of Early Childhood Concerns Substance Abuse and Mental Health Services Administration Rockwall II, Room 950 5600 Fishe-rs Lane Rockville, MD 20857 P: (301) 443-0519 F: (301) 443-7878 Dr. Wallace Mandell Professor Department of Mental Hygiene School of Hygiene and Public Health The Johns Hopkins University 3932 Cloverhill Road Baltimore, MD 21218 P: (410) 955-3889 Ms. Phyllis E. Marshall Consultant 1375 Walnut Avenue Annapolis, MD 21403 P: (410) 280-6774 F: (410) 280-6774 Dr. John Morgan Director Clinical and Prevention Services Chesterfield Mental Health Mental Retardation, and Substance Abuse Services P.O. Box 92 Chesterfield, VA 23832 P: (804) 768-7201 F: (804) 768-9205 Dr. Patricia Mrazek Consultant 7107 Laverock Lane Bethesda, MD 20817 P: (301) 320-0045 F: (301) 320-5740 Dr. Ricardo Munoz Professor Department of Psychiatry University of California, San Francisco Suite 7M 1001 Potrero Avenue San Francisco, CA 34110 P: (415) 206-5214 F: (415) 206-8942 Ms. Wilma Pinnock Program Assistant Office of Managed Care Center for Substance Abuse Prevention Substance Abuse and Mental Health Services Administration Rockwall II, Suite 901 5600 Fishers Lane Rockville, MD 20857 P: (301) 594-0788 F: (301) 443-1548 Dr. Mark Publicker Chief, Addiction Medicine Kaiser, Mid-Atlantic Region Kaiser Permanente Suite 300 8550 Lee Highway Merrifield, VA 22031 P: (703) 207-2853 F: (703) 207-2838 Ms. Gail Ritchie Public Health Adviser Special Programs Development Branch Center for Mental Health Services Substance Abuse and Mental Health Services Administration Parklawn Building, Room 18C-07 5600 Fishers Lane Rockville, MD 20857 P: (301) 443-7790 F: (301) 443-7912 Dr. Elizabeth Robertson Team Leader Prevention Research Branch Division of Epidemiology and Prevention Research, National Institute on Drug Abuse Parklawn Building, Room 9A-53 5600 Fishers Lane Rockville, MD 20857 P: (301) 443-1514 F: (301) 443-2636 Mr. James Robinson Vice President of Special Programs Managed Healthcare Systems Fifth Floor 7 Hanover Square New York, NY 10004 P: (212) 509-5999 F: (212) 509-2966 Ms. Sheri Rucker Senior Public Health Advisor Office of Managed Care Center for Substance Abuse and Mental Health Services Administration Rockwall II, Suite 901 5600 Fishers Lane Rockville, MD 20857 P: (301) 594-0788 F: (301) 443-1548 Mr. Philip Salzman Director of Prevention and Managed Care Advocates for Human Potential, Inc. 323 Boston Post Road Sudbury, MA 01776 P: (508) 443-0055, ext. 226 F: (508) 443-4722 Ms. Clare Sharda Director of Accreditation Policy National Committee for Quality Assurance Suite 500 2000 L Street, NW Washington, DC 20036 P: (202) 955-3517 F: (202) 955-3599 Ms. Shelagh Smith Senior Public Health Adviser Office of Managed Care Center for Mental Health Services Substance Abuse and Mental Health Services Administration Parklawn Building, Room 15-105 5600 Fishers Lane Rockville, MD 20857 P: (301) 443-4782 F: (301) 443-1563 Dr. Gail Stuart Professor Colleges of Nursing and Medicine Center for Health Care Research Medical University of South Carolina Strom Thurmond Research Building Room 630 171 Ashley Avenue Charleston, SC 29425 P: (803) 953-6617 F: (803) 792-5567 Ms. Betty Tableman Director Prevention Services 1515 Moores River Drive Lansing , MI 48910 P: (517) 335-0124 F: (517) 335-2667 Ms. Christina Thompson National EAP Director Green Spring Health Services, Inc. 5565 Sterrett Place Columbia, MD 21044 P: (410) 964-8019 F: (410) 740-8573 Dr. Nancy Valentine Chief Consultant Nursing Strategic Healthcare Group Department of Veterans Affairs 810 Vermont Avenue, NW Washington, DC 20420 P: (202) 273-8421 F: (202) 273-9066 Mr. Charles Williams Senior Public Health Adviser Office of Managed Care Center for Substance Abuse Prevention Substance Abuse and Mental Health Services Administration Rockwall II, Suite 901 5600 Fishers Lane Rockville, MD 20857 P: (301) 443-5254 F: (301) 443-5254 Ms. Irene Wozny National Mental Health Association Board of Directors 2121 East Baltimore Street Baltimore, MD 21231 P: (410) 767-4100 F: (410) 333-7097 PROCESS: Dr. Neal Adams and Dr. Eric Goplerud. Moderators MEMBERS: STRUCTURE: Dr. Richard Elpers and Mr. John Morris, Moderators MEMBERS: Program Chair and Editor: John Morris IX. APPENDICES AND TABLES 1 Family is defined broadly to include relatives who are primary caregivers of youth and other guardians who provide a family environment (e.g., adoptive families, foster care families). 1 Revised 1/26/98 DRAFT: NOT FOR CITATION OR DUPLICATION