Concept Paper

What Was Learned About Integrated Services and Their Evaluation within the HRSA/HAB/SPNS Cooperative Agreements (1994 – 1999)

George J. Huba, Ph.D.[1]
The Measurement Group

 

The HRSA/HAB/SPNS Cooperative Agreements were funded between October 1994 and September 1999. The initial set of projects consisted of 27 grants to develop innovative HIV/AIDS services. Grantees were funded to develop managed medical care models, develop integrated university-based care models, develop community based organization models for supporting medical care, develop methods for enhancing the infrastructure for HIV/AIDS services, or develop innovative training models for healthcare providers. Descriptions of the programs and materials deriving from those projects are given online at www.TheMeasurementGroup.com/edc_main.htm.

This Concept Paper primarily consists of two sets of materials that appear elsewhere.  

  • The first source is Chapter 18 of the book, Outcomes Achieved from the Provision of Innovative HIV/AIDS Services: The HRSA/HAB SPNS Cooperative Agreements, 1994—1999, edited by G. J. Huba, Ph.D., Lisa A. Melchior, Ph.D., A. T. Panter, Ph.D., and Barbara Aranda-Naranjo, Ph.D., Haworth Press, in press. Chapter 18 is authored by Huba, Melchior, Panter, and Aranda-Naranjo. The materials here are expanded upon in that chapter. To be put on a mailing list for copies of the final manuscript, send an email to ghuba@TheMeasurementGroup.com.
     
  • The second source of information is The Measurement Group Knowledge Base on HIV/AIDS Care (Huba, G. J., Melchior, L. A., Panter, A. T., 1998-2001, online at www.TheMeasurementGroup.com). Selected links to information are given in relevant sections below. The more general link to the main index for the Knowledge Base is www.TheMeasurementGroup.com/KnowBase/CoopAgree/knowbaseca.htm. Certain key findings from the entire Knowledge Base are summarized at www.TheMeasurementGroup.com/KnowBase/CoopAgree/Bullets.htm.

Lessons Learned about Integrated or Wrap-Around Services from the Cooperative Agreements

Wrap-around or integrated psychosocial services significantly increase the positive outcomes that can be obtained with patients in HIV/AIDS programs. Many different results using different segments of the Cooperative Agreement data and differing analytic techniques converge to show the advantages of wrap-around services, especially for those patients with high levels of service need and vulnerability from populations that have traditionally been underserved in the medical system.

  1. Medical indicators – CD4 counts, viral load – improved over time for all medical projects in aggregate. For viral load, the change is somewhat larger when wrap-around services are provided. Note that our evaluations did not have access to good measures of patient adherence to treatment recommendations and that these projects were conducted during the introduction period for current pharmacologic therapies. Key findings are provided online at www.TheMeasurementGroup.com/KnowBase/CoopAgree/MedicalOutcomes/14/knowledgeitem.htm and www.TheMeasurementGroup.com/KnowBase/CoopAgree/MedicalOutcomes/22/knowledgeitem.htm with additional results on clinically significant degrees of change over time provided online at  www.TheMeasurementGroup.com/KnowBase/CoopAgree/MedicalOutcomes/49/KnowledgeItem.htm and www.TheMeasurementGroup.com/KnowBase/CoopAgree/MedicalOutcomes/52/KnowledgeItem.htm.

  2. Patients who received wrap-around services also had greater positive changes in self-rated, health-related quality of life over time, while those individuals who received medical services from managed care providers without the same degree of wrap-around services did not have increased self-rated quality of life over time. Key findings are provided online at www.TheMeasurementGroup.com/KnowBase/CoopAgree/QualityOfLife/39/KnowledgeItem.htm and www.TheMeasurementGroup.com/KnowBase/CoopAgree/QualityOfLife/60/KnowledgeItem.htm.

  3. Patients who received wrap-around services also had a greater decrease in self-reported symptom severity over time than did those who did not. Key findings are provided online at www.TheMeasurementGroup.com/KnowBase/CoopAgree/QualityOfLife/61/KnowledgeItem.htm.

  4. Among the individuals who received wrap-around services, self-rated barriers to receiving medical and other needed services dropped significantly. Key findings are provided online at www.TheMeasurementGroup.com/KnowBase/CoopAgree/ClientBarriers/30/knowledgeitem.htm.

  5. Among the individuals who received wrap-around services, self-reported levels of psychological distress (depression) significantly decreased. Key findings are provided online at www.TheMeasurementGroup.com/KnowBase/CoopAgree/ClientDistress/14/knowledgeitem.htm.

  6. Among the individuals who received wrap-around services, self-reported levels of unmet service needs significantly decreased. Key findings are provided online at www.TheMeasurementGroup.com/KnowBase/CoopAgree/ClientUnmetNeeds/9/knowledgeitem.htm.

  7. Patients who received services from either an integrated medical services provider (within an University-based clinic) or from a Community Based Organization providing supports to patients of linked medical services providers were more satisfied with their overall service quality than were patients who received services from a managed medical care provider. Key findings are provided online at www.TheMeasurementGroup.com/KnowBase/CoopAgree/ClientSatisfaction/19/knowledgeitem.htm.

  8. Case management significantly impacted upon program retention and the time until the receipt of other services. Retention in programs was much greater for those patients who received case management, especially those who received the service early in the overall treatment episode. Key findings are provided online at www.TheMeasurementGroup.com/KnowBase/CoopAgree/CaseManagement/1/KnowledgeItem.htm and www.TheMeasurementGroup.com/KnowBase/CoopAgree/CaseManagement/6/KnowledgeItem.htm.

  9. Organizational studies of three University-based integrated service providers suggested a number of strategies for implementing and managing such services. Those studies are presented online at www.TheMeasurementGroup.com/KnowBase/CoopAgree/Index_Special/CHWG.htm and www.TheMeasurementGroup.com/KnowBase/CoopAgree/Index_Special/Interviews.htm and www.TheMeasurementGroup.com/tmg/reports/compcare/com_intro.htm. In aggregate, these three online resources show major strategies for implementing such programs, lessons learned, key observations from staff and observers, and recommendations for replicating the successful program elements.

Indicators of Outcomes from Integrated Services

Evaluation materials used in the Cooperative Agreement Projects are stored online. The following links are to the key questionnaires and other materials used to study the outcomes. The use of the instruments is described in the Data Source sections of the online Knowledge Items where they are employed.

The general index for all evaluation modules – questionnaires, surveys, other methods – used in the cross-cutting evaluation of the Cooperative Agreement projects is given online at www.TheMeasurementGroup.com/mod_list.htm and more specifically at www.TheMeasurementGroup.com/modgrid.htm. Modules specifically used to collect outcome indicators are listed below.

In additional to the actual data collection instruments, the Knowledge Base contains a number of analyses for each of these data modules that show basic psychometric characteristics (reliability, internal consistency) and results that may permit individual instruments to be shortened or otherwise altered for future use.

Indicators of Quality of Integrated Services

Indicators for the quality of integrated services can be inferred and developed from the series of online knowledge items in The Measurement Group Online Knowledge Base on HIV/AIDS Care on psychosocial services. All projects providing psychosocial services – 39 specific services which may be categorized into five broad categories (two of which have two subcategories for a total of nine major themes) – were either University-based integrated care providers or Psychosocial Community Based Organizations working to support a medical services provider. The coding scheme for the psychosocial services is given at: http://www.TheMeasurementGroup.com/Definitions/Psychosocial%20Services.htm. The form used to code psychosocial services is stored online at: www.TheMeasurementGroup.com/modules/module2.htm. Examining these two documents shows the services that the Cooperative Agreement projects felt would be necessary include in a continuum of psychosocial services. Models of the relationship. The following list shows the major types of psychosocial services and how they can be coded into broader categories.

HIV Prevention/Intervention Services

  • HIV risk assessment
  • HIV pre-test counseling
  • HIV testing
  • HIV post-test counseling
  • HIV prevention
  • Other HIV-related services

Category has remained as originally coded

Mental Health Services

  • Individual therapy/counseling
  • Psychiatric evaluation
  • Psychosocial assessment
  • Crisis intervention (individual)
  • Other mental health

Reassigned as follows:

  • Individual Counseling Services = individual therapy/counseling OR individual crisis intervention OR other mental health
  • Psychiatric/Psychosocial Assessment = psychiatric evaluation OR psychosocial assessment

Group Counseling

  • Crisis intervention (group)
  • Family/couple counseling
  • Group counseling
  • Peer support group
  • Other group

Reassigned as follows:

  • Group (not Couple) Counseling = group crisis intervention OR group counseling OR peer support group
  • Family/Couple counseling = family/couple counseling

Substance Abuse Services

  • 12-step group
  • Relapse prevention
  • Substance abuse counseling

Category has remained as originally coded

Case Management Services

  • Advocacy services
  • Clinical assessment
  • Educational services
  • Financial services
  • Housing
  • Legal services
  • Medical
  • Vocational services
  • Other case management

Category has remained as originally coded  

A set of 37 Knowledge Items is presented online that show many different characteristics of the services provided by these national demonstration projects using methods considered state-of-the-art between 1994 and 1999. These Knowledge Items look at the overall service histories of 2063 HIV/AIDS patients on the psychosocial (wrap-around) services provided at 10 providers, six of which were Psychosocial CBOs and four of which were University-based integrated care providers.

We believe that an overall set of recommendations on quality standards for integrated services can be developed, in part, based on the patterns of services that these providers found it necessary to provide to their patients.

An index of the 37 Knowledge Items on patterns of psychosocial services is given online at http://www.TheMeasurementGroup.com/KnowBase/CoopAgree/Index_Client/PsychosocialServices.htm.

 

v v v v v

The following table shows information provided in these 37 Knowledge Items that may be useful for developing indicators of the quality of integrated services. Note that the use of this information requires the assumption that these projects were providing high quality care, tailored to the needs of the patient within these national demonstration programs. We believe that the overall pattern of statistical studies on the outcomes of the patients of these programs suggests that state-of-the-art services for high-need, vulnerable groups of individuals living with HIV/AIDS were provided.

Summary Table Linking Individual Knowledge Items on Psychosocial Services to Relevance to Developing Quality of Service Indicators

Knowledge Item

URL

Topic

Relevance to Developing Quality of Service Indicators

1

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/1/KnowledgeItem.htm

 

Presence or absence of service utilization in each of nine categories by time in program and whether or not still active at end of funding period

This Knowledge Item shows the percentage of patients – in aggregate over 10 projects – who were provided each type of service at least once in their treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item.

2

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/2/KnowledgeItem.htm

Presence or absence of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and race-ethnicity

This Knowledge Item shows the percentage of patients – in aggregate over 10 projects – who were provided each type of service at least once in their treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item.

3

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/3/KnowledgeItem.htm

Presence or absence of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and age group

This Knowledge Item shows the percentage of patients – in aggregate over 10 projects – who were provided each type of service at least once in their treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item.

4

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/6/KnowledgeItem.htm

Presence or absence of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and gender

This Knowledge Item shows the percentage of patients – in aggregate over 10 projects – who were provided each type of service at least once in their treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item.

5

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/5/KnowledgeItem.htm

Presence or absence of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and specific project

This Knowledge Item shows the percentage of patients – within each specific project – who were provided each type of service at least once in their treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item which also shows the approximate range of differences among state-of-the-art programs that might be expected.

6

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/6/KnowledgeItem.htm

Presence or absence of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and race-ethnicity within specific project

This Knowledge Item shows the percentage of patients – within each specific project – who were provided each type of service at least once in their treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item which also shows the approximate range of differences among state-of-the-art programs that might be expected.

7

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/7/KnowledgeItem.htm

Presence or absence of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and age group within specific project

This Knowledge Item shows the percentage of patients – within each specific project – who were provided each type of service at least once in their treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item which also shows the approximate range of differences among state-of-the-art programs that might be expected.

8

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/8/KnowledgeItem.htm

Presence or absence of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and gender within specific project

This Knowledge Item shows the percentage of patients – within each specific project – who were provided each type of service at least once in their treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item which also shows the approximate range of differences among state-of-the-art programs that might be expected.

9

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/9/KnowledgeItem.htm

Presence or absence of service utilization in each of nine categories assessing simultaneous effects of time in program, whether or not still active at end of funding period, gender, race-ethnicity, age group, specific project

This Knowledge Item shows the range of expected differences among 10 state-of-the-art programs in terms of whether a patient will receive a specific type of service or not.

10

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/10/KnowledgeItem.htm

Days of service utilization in each of nine categories by time in program and whether or not still active at end of funding period

This Knowledge Item shows the number of service episodes – in aggregate over 10 projects – for each type of service provided to patients during the treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item.

11

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/11/KnowledgeItem.htm

Days of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and race ethnicity

This Knowledge Item shows the number of service episodes – in aggregate over 10 projects – for each type of service provided to patients during the treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item.

12

http://www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/12/KnowledgeItem.htm

Days of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and age group

This Knowledge Item shows the number of service episodes – in aggregate over 10 projects – for each type of service provided to patients during the treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item.

13

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/13/KnowledgeItem.htm

Days of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and gender

This Knowledge Item shows the number of service episodes – in aggregate over 10 projects – for each type of service provided to patients during the treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item.

14

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/14/KnowledgeItem.htm

Days of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and specific project

This Knowledge Item shows the number of service episodes – within each of 10 projects – for each type of service provided to patients during the treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item. The Knowledge Item also shows the variation among projects that would be expected.

15

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/15/KnowledgeItem.htm

Days of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and race-ethnicity within specific project

This Knowledge Item shows the number of service episodes – within each of 10 projects – for each type of service provided to patients during the treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item. The Knowledge Item also shows the variation among projects that would be expected.

16

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/16/KnowledgeItem.htm

Days of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and age group within specific project

This Knowledge Item shows the number of service episodes – within each of 10 projects – for each type of service provided to patients during the treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item. The Knowledge Item also shows the variation among projects that would be expected.

17

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/17/KnowledgeItem.htm

Days of service utilization in each of nine categories by time in program and whether or not still active at end of funding period and gender within specific project

This Knowledge Item shows the number of service episodes – within each of 10 projects – for each type of service provided to patients during the treatment episode. Minimum levels of expected level of service provision standards are suggested by this Knowledge Item. The Knowledge Item also shows the variation among projects that would be expected.

18

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/18/KnowledgeItem.htm

Days of service utilization in each of nine categories assessing simultaneous effects of time in program, whether or not still active at end of funding period, gender, race-ethnicity, age group, specific project

This Knowledge Item shows the range of expected differences among 10 state-of-the-art programs in terms of the number of days of services of a specific type that will be provided to a patient.

19

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/19/KnowledgeItem.htm

Presence or absence of service utilization in each of nine categories assessing simultaneous effects of time in program, whether or not still active at end of funding period, gender, and total service needs-vulnerabilities

This Knowledge Item shows how the receipt of a specific service is related to total patient service needs, vulnerabilities, and demographic factors.

20

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/20/KnowledgeItem.htm

Days of service utilization in each of nine categories assessing simultaneous effects of time in program, whether or not still active at end of funding period, gender, and total service needs-vulnerabilities

This Knowledge Item shows how the total dosage of services is related to total patient service needs and vulnerabilities.

21

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/21/KnowledgeItem.htm

Inter-relationship of the service utilization rates for major categories of services

This Knowledge Item uses a variety of psychometric and data analytic methods to study the rates at which patients received different types of services during a typical psychosocial services episode to empirically define the services that the programs felt it was necessary to provide to the modal high need patient. Rates of receiving services fall into several major clusters.

22

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/22/KnowledgeItem.htm

Relationship of total time in the program and the number of service days

This Knowledge Item can be used to show the relationship between time in program and expected total number of service dates.

23

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/23/KnowledgeItem.htm

Statistical models relating utilization of each of nine types of services to service needs and vulnerabilities and other factors

This Knowledge Item shows how service patterns may be related to service needs, vulnerabilities, and demographic factors.

24

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/24/KnowledgeItem.htm

Statistical model relating the days of services to service needs and vulnerabilities and other factors

This Knowledge Item shows how service patterns may be related to service needs, vulnerabilities, and demographic factors.

25

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/25/KnowledgeItem.htm

Relationship of status as a drug abuser to service utilization.

This Knowledge Item suggests how service patterns may be related to status as a substance abuser with HIV/AIDS.

26

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/26/KnowledgeItem.htm

Statistical models showing the relationship between different topics that may be discussed with the patient during the service episode.

This Knowledge Item suggests certain categories of topics that may need to be addressed with a patient (in one or more specific treatment modalities) during a treatment episode.

27

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/27/KnowledgeItem.htm

Statistical models show the likelihood of service utilization based on the number of days in the program for each of 10 projects.

This Knowledge Item suggests possible standards for how likely it is that a service will be provided to a patient given different lengths of stay in the program. Variations among programs are illustrated.

28

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/28/KnowledgeItem.htm

Statistical models show the likelihood of service utilization based on the number of days in the program and differing numbers of patient needs-vulnerabilities.

This Knowledge Item suggests possible standards for how likely it is that a service will be provided to a patient given different lengths of stay in the program and differing need-vulnerability levels.

29

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/29/KnowledgeItem.htm

Statistical models show the likelihood of service utilization based on the number of days in the program and drug abuse status.

This Knowledge Item suggests possible standards for how likely it is that a service will be provided to a patient given different lengths of stay in the program and differing patterns of substance abuse.

30

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/30/KnowledgeItem.htm

Statistical models show the likelihood of service utilization based on the number of days in the program and whether the program was a CBO providing psychosocial supports or a medical provider with enhanced psychosocial supports.

This Knowledge Item suggests possible standards for how likely it is that a service will be provided to a patient given different lengths of stay in the program and whether the program was a CBO providing psychosocial supports or a medical provider with enhanced psychosocial supports.

31

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/31/KnowledgeItem.htm

Psychometric models show the likelihood of utilizing each of seven major types of services during the service episode.

This Knowledge Item permits programs to be compared in terms of their overall service coverage when the programs compared do not provide the exact same array of services (that is, the types of services provided do not overlap 100% among the programs compared).

32

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/32/KnowledgeItem.htm

Psychometric models show the likelihood of utilizing each of 39 specific types of services during the service episode.

This Knowledge Item permits programs to be compared in terms of their overall service coverage when the programs compared do not provide the exact same array of services (that is, the types of services provided do not overlap 100% among the programs compared).

33

www.TheMeasurementGroup.com/KnowBase/
CoopAgree/PsychosocialServices/33/KnowledgeItem.htm