Glossary
Components of the Evaluation
The five study components that comprise the national evaluation are discussed below. Sites are most directly involved in the collection of data for the Cross-sectional Descriptive Study, the Child and Family Outcomes Study, and the Practice Level Assessment. The remaining study components involve data collection that occurs primarily at the system level by the national evaluation team with support from site personnel. Aggregate data about each component, including data collection instruments used, can be located at the Purdue University Calumet web site:
Cross-sectional
Descriptive Study
The purpose of this study is to describe the children enrolled in the funded system of care in terms of their demographics, functional status, living arrangement, diagnosis, risk factors, and mental health service history. Family demographics, socioeconomic status, and composition will also be described.
These data are collected for all children and families as they enter services in systems of care and comparison sites throughout the program's funding period. In addition, for the children and families who participate in the Child and Family Outcome Study, the descriptive data elements that may have changed over time (e.g., family income legal custody) are collected again at follow-up data collection points.
Child and Family Outcome Study
The purpose of this study is to examine how participation in services in a system of care affects child clinical and functional status and family life. Outcome data on child clinical and functional status will be used to assess change over time in the following areas: symptomatology, diagnosis, social functioning, substance use, school attendance and performance, delinquency and juvenile justice involvement, and stability of living arrangements. Family life will be assessed in the areas of family functioning, caregiver strain, and material resources. These data are collected within 30 days of the child's entry into services and at 6-month intervals for the length of the evaluation.
Practice Level Assessment
This study component is designed to assess the degree to which system of care values and principles have been incorporated into service delivery and practice. The Practice Level Assessment includes the collection of information on services received through different service sectors and satisfaction with services. The Practice Level Assessment is a part of each follow-up interview conducted with families enrolled in the Child and Family Outcome Study. All families enrolled in the outcomes study participate in the Practice Level Assessment, and no additional eligibility criteria must be met for participation in this assessment.
System
Level Assessment
The purpose of this study is to examine whether programs have been implemented in accordance with the system of care philosophy and to document how systems develop over time to meet the needs of the children and families they serve. Of particular interest is whether services are delivered in an individualized, family focused, coordinated manner, and whether the system involves multiple child-serving agencies.
Data collection occurs primarily during site visits by the national evaluation team, which are scheduled to take place every 18 months beginning in the second grant year. Information will be collected through a combination of pre-site visit document review of randomly selected case records, and semi-structured interviews. Categories of respondents will include project directors, agency representatives, direct service providers, case coordinators/managers, and family members.
Services and Costs Study
The Purpose of this study is to describe the types of services used by children and families, their utilization patterns, and the associated costs. The relationship between service and outcomes is also explored. This study draws on data maintained in fiscal (e.g., charge, billing) management information systems (MIS's) within grantee communities. After an initial mailed survey, a determination is made as to the appropriate level of involvement of each site in this study component. Depending upon the nature of the local MIS's, a site-specific plan is devised for the transmission of appropriate data. Of interest are the types of services, the combination of services, continuity or gaps in care, the length of treatment, and service and system costs. This information will be analyzed with the data collected from children and family members to create a comprehensive picture of the services and supports families use.
