Four Winds Hospitals

The Effectiveness of Mental Health Treatment
Clinical Outcome Study
Four Winds Saratoga

For more than 15 years Four Winds Saratoga has evaluated the effectiveness of the clinical treatment provided to our patients. We have studied how well our treatment works and have used this research to make clinical improvements to our programs. This Clinical Outcome Research is ongoing and we are committed to continuing to use Outcome Measurement to improve the quality of our treatment.

Components of Effective Treatment
1. Symptom Reduction
The immediate reduction of the symptoms that precipitated an admission

2. Maintenance of Treatment Gains
The extent to which the changes that occurred in the inpatient stay are maintained after discharge from the hospital

3. After-Care Follow-Up
The extent to which the individual followed-up with the aftercare recommendations

4. Consumer Satisfaction
Consumer satisfaction is the final important component of the effectiveness of treatment. If people are satisfied with their treatment, they are more likely to follow through with treatment recommendations. Many providers who study outcomes examine only satisfaction. Although satisfaction is very important, a satisfied consumer who experiences no changes in their symptoms or functional status is not a treatment success. The crucial measure of satisfaction is the report of the consumer, during and after they have received their treatment.

How do we Measure Effectiveness?

The ideal way to study effectiveness is to examine every patient with an extensive assessment. This is not always possible, because of the large number of patients who are seen at Four Winds (over 2000 per year). However, over the past four years 80% of our patients have participated in outcome and satisfaction studies.

Our practice is to obtain information from patients, family members and clinicians at admission, discharge, and 30 days post-discharge. With specialized staff collecting the information, patients provide detailed information about their treatment and how they are doing after discharge. This information includes ratings of symptoms, functioning and satisfaction by the therapist, the patient, family member and the outcome staff.

At this time, Four Winds Saratoga has accumulated ratings for over 10,000 persons who have been treated at our facility since 1996. We have used this information to look at the four areas of outcome as described above. Our commitment to outcome evaluation has led to the development of this substantial database.

How Effective is Treatment at Four Winds Hospital?

Symptom Reduction

For this component Four Winds Saratoga examined the changes in symptoms during patients' treatment. Global clinical ratings of symptoms at admission and discharge (Derogatis Psychiatric Rating Scale) were obtained from the clinical setting and from the outcome staff, for 791 adult inpatients, 540 child/adolescent inpatients and 265 Partial Hospital and Intensive Outpatient Program patients. Using the Global Assessment of Functioning (GAG), outcome staff and therapists also rated 821 adults and 876 children and adolescents who received inpatient treatment at Four Winds during the year 2000.

As you can see in Figures 1-5, therapy and outcome staff found that the patients who were treated improved significantly in both clinical symptoms and overall functional status during their treatment.

Figures 1 - 5

Maintenance of Treatment Gains

For those patients who agreed to participate, a 30 day post-discharge follow-up assessment was conducted. This self-report assessment included examinations of the same symptoms that were assessed at admission and discharge as well as ratings of their functional status. As shown in Figures 6 and 7, from the perspective of both clinical symptoms and overall functional status, the treatment gains experienced during the inpatient stay persisted over a 30 day follow-up period. This data suggests that the treatment program provides a sustained benefit.

Figures 6-7

The Role of Discharge Planning

If a patient did very well and followed their discharge plan, this would suggest a strong relationship between the discharge plan and the positive outcome. Discharge plans for both children/adolescents and adults generally included both psychotherapy and medication.

Figure 8 shows how many of the child/adolescent and adult patients participated in
outpatient therapy and took their medications post discharge. As you can see, most patients were still participating in the discharge plan as advised after leaving the hospital. While high
levels of compliance with psychotherapy may not be surprising, the level of medication
compliance is higher than typically reported in studies of treatment compliance.

Figure 8

How Satisfied are the People who were treated at Four Winds?

Using a comprehensive survey we asked patients to rate their satisfaction with their treatment at Four Winds Saratoga. Adult inpatients, child/adolescent inpatients and their parents, and Partial Hospital and Intensive Outpatient Program patients were given the survey. As you can see from Figures 9-12, there was an overwhelmingly positive response to the treatment provided to patients at Four Winds.

We specifically asked former adult patients and parents of child and adolescent patients, if they were satisfied enough with our treatment to consider returning if their (or their children's) problems re-occurred - more than 96% said yes. We also asked them if they were satisfied enough with our services to recommend us to a friend or relative who was in need of psychiatric care - more than 96% said yes.

Children who took the survey were also overwhelmingly satisfied with the treatment they received - 90% of them reported that they were helped with their problems while at Four Winds.

Figures 9 - 12

What Do We Do With These Data?

There are many uses of the outcome data collected at Four Winds, the most important of which is to establish a continuous process of evaluating the care that is provided at the hospital. Regular presentations and workshops about our outcome practice are presented to the staff at Four Winds. We have run workshops on effective diagnosis, accurate clinical assessment, and compliance with mutually agreed upon treatment for adolescent and adult services.

This information is also used to demonstrate to accreditation agencies that we are continuing to improve the care that we provide. For instance, the Joint Commission on the Accreditation of Health Care Organizations (JCAHO) monitors the extent to which quality assurance efforts are continuously translated into practice. Four Winds Saratoga's outcome study was published in the Joint Commission's manual entitled, Using Performance Measurement to Improve Outcomes in Behavioral Health Care, providing a guide to the development and use of performance measures in behavioral health care.

Four Winds Saratoga has undertaken a series of more issue-specific studies. The goal of these studies is both to aid the Quality Improvement process and also to identify and develop models for studies in which outcome data can be brought to bear in more specific and focused ways that are responsive to the immediate needs of the setting. Each of these studies serves as an example of how such projects can be devised and executed. Each started with outcome data, and used this data to confirm or disconfirm impressions regarding problems within the system, and to serve as a benchmark against which to evaluate the impact of quality improvement interventions.

For further information about Four Winds Clinical Outcome Research, please contact Lynne Knauf, C.S.W., Director of Quality Management and Director of Child and Adolescent Services, Four Winds Hospital.

The following is a list of the papers written and abstracts published by the outcome study team at Four Winds.

Papers Previously Published:

Hoffmann, F.L., Leckman, E., Russo, N. & Knauf, L. (1999) In it for the long haul: The integration of outcomes assessment, clinical services, and management decision-making. Journal of Evaluation and Program Planning 22:211-219.

Hoffmann, F.L., Capelli, K. & Mastrianni, X (1997) Measuring treatment outcome for adults and adolescents: Reliability and validity of BASIS-32. Journal of Mental Health Administration 24(3):316-331.

Hoffmann, F.L. & Mastrianni, X. (1995) Partial hospitalization following inpatient treatment: Patient characteristics and treatment outcome. Continuum: Developments in Ambulatory Mental Health Care (formerly International Journal of Partial Hospitalization) 2(4): 247-263.

Hoffmann, F.L. & Mastrianni, X. (1993) The role of supported education in the inpatient treatment of young adults: A two-site comparison. Psychosocial Rehabilitation Journal 17(1):109-120.

Hoffmann, F.L. & Mastrianni, X. (1992) The hospitalized young adult: New directions for psychiatric treatment. The American Journal of Orthopsychiatry 61(2):97-102.

Hoffmann, F.L. & Mastrianni, X. (1991) Psychiatric leave policies: Myth and reality. Journal of College Student Psychotherapy 6(2):3-20.

Hoffmann, F.L. & Mastrianni, X. (1991) The authors respond (Responses to Arnstein, Liptzin, Marsh and May's Commentaries.) Journal of College Student Psychotherapy 6(2): 53-57.

Hoffmann, F.L. & Mastrianni, X. (1989) The mentally ill student on campus: Theory and practice. Journal of American College Health 38(1):15-20.