Evaluating System Dynamics Group Model Building in Public Service System Change

Principal Investigator: Steven Huz, M.P.A., Co-investigators: David Andersen, George Richardson, Roger Boothroyd, Marian Schwager, and Roberta Spencer

PROJECT GOALS

This study examines which strategies are most effective in increasing interagency collaboration and improving the coordination of vocational rehabilitation services for adult persons with severe mental disorder (PSMD). The project seeks to determine the extent computer-based system dynamics modeling, combined with facilitated group activities, enhances the coordination and integration of local vocational rehabilitation and mental health services for PSMD. In addition to testing the efficacy of group model building as a facilitator of mental health system change, the project examines the impact of a services integration demonstration project (SIP). The SIP initiative gives county service providers an explicit mandate to work toward integration of mental health and vocational rehabilitation services. The project also seeks to develop an enhanced capacity for providing group model building and "system thinking" support to local service providers in constructing more efficient service delivery systems.

RESEARCH ACTIVITIES AND RESULTS

To examine the impact of group model building for public service system integration, researchers used a three group design. The three study conditions included: (1) two SIP county teams who received the group model intervention; (2) two SIP county teams who did not receive the group model building intervention; and (3) two teams of service providers and policy makers in non-SIP counties receiving the group model building intervention.

The six counties were matched on geographic characteristics and the maturity of the local services integration effort. Data were collected pre- and post-intervention on group member characteristics, attitudes and beliefs about the importance of integrating services, organizational networks, and perceptions of the value and effectiveness of the intervention. A standardized group model building intervention and project instrumentation were developed through a pilot conducted in Westchester County.

Group model building interventions and pre- and post-intervention data collection has been completed in four study sites. In addition, comparable data were collected in two sites where no group model building was administered. The study sites were: Schenectady and St. Lawrence Counties (non-SIP, group model building), Nassau and New York Counties (SIP, group model building, and Kings and Erie Counties (SIP, no group model building).

Summary of Findings: Participants from sites where group model building occurred reported greater progress in creating processes to support system change in their service system than did policy makers and providers in counties where there was no group model building. Analyses of organizational network data collected in group model building sites showed significant shifts on aspects of organizational relationships which contribute to a more cohesive system. On measures of team functioning (problem importance, problem understanding, alignment, goal clarity, team cohesion, internal fragmentation, openness of group process) individuals from group model building sites reported significant improvements, while individuals from sites where no group model building occurred reported no shift.

POLICY IMPLICATIONS/ SIGNIFICANCE OF FINDINGS

In all sites where the group model building intervention was administered concrete evidence of increased integrative activities were apparent. Examples of these include:

development and publication of county employment opportunity guides;
development of coordinated training curricula for local vocational rehabilitation and mental health provider staff;
development of coordinated admission and assessment forms in use by both systems,
hiring of a mental health/vocational rehabilitation service system coordinator to assist service recipients; and
establishment of ongoing efforts to promote and enhance communication and coordinated service system planning.

PLANS

Plans include continued dissemination of project findings through submission of journal articles to publications aimed at the mental health services research, organizational development, and system thinking communities.

Presentations:

Richardson, G.P., Andersen, D.F., Huz, S., Boothroyd, R. and Spencer, R. (1998). Outcomes from an Evaluation of a Group Model Building Intervention for Public Sector System Change. Paper presented at Annual Meeting of the International System Dynamics Society, Quebec City, Quebec, July 1998

Huz, S., Banks, S. and Boothroyd, R. (1996) Scaling and Clustering Analytic Techniques to Assess Group Consensus from a System Thinking Intervention to Promote Mental Health System Change. Paper presented at Annual Meeting of the American Public Health Association, New York, NY. November 1996.

Huz, S., Andersen, D. F., Richardson, G. P., & Boothroyd, R.A. (1996) Evaluating Group Model Building. Paper presented at 1996 Information, Operations Research, Management Science (INFORMS) Conference, Washington DC.

Huz, S., and Boothroyd, R.A.(1996) Improving Vocational Rehabilitation Service Delivery to Individuals with Severe Mental Illness: Researching a Systems Integration Strategy. Paper presented at 1996 National Association of State Mental Health Program Directors (NASMHPD) Conference, Arlington, VA.

Huz, S., Boothroyd, R.A., Schwager, M., Andersen, D.F., and Richardson, G.P. (1995). A Services Integration Strategy to Improve Vocational Rehabilitation Service Delivery to Individuals with Severe Mental Illness. Paper presented at Annual Meeting of the American Public Health Association, San Diego, CA.

Huz, S. Andersen, D.F., Richardson, G.P. and Boothroyd, R.A. (1995). Measuring Mental Model Alignment from Group Model Building. Paper presented at 1995 International System Dynamics Conference, Tokyo Japan.

Publications:

Andersen, D. F. and Richardson, G.P. (1997) Scripts for Group Model Building. System Dynamics Review. 13, 2, 107-129.

Huz, S., Richardson, G.P., Andersen, D.F., and Boothroyd, R.A., (1997) A Framework for Evaluating Systems Thinking Interventions: An Experimental Approach to Mental Health System Change. System Dynamics Review, 13, (2), 149-169.

Project Completed.

[Top]