Investigating the Degree of Reliable Change Among Persons Assigned to Receive Mental Health Peer Specialist Services
Abstract
Objective:
Peer specialists are individuals with mental illness and substance use disorders trained to use their experiences to help others with similar disorders. Evidence for the effectiveness of peer specialist services has been mixed in previous randomized trials using intent-to-treat analyses, possibly because of variation in the intensity of treatment delivered. This study, which was part of a larger randomized trial, assessed whether level of peer specialist engagement was associated with reliable positive change on measures of psychiatric symptoms and hope.
Methods:
The Reliable Change Index was used to compute whether veterans (N=140) achieved reliable positive change on standardized baseline-to-posttest assessments of psychiatric symptoms and hope. Logistic regression analyses were conducted to predict positive change in symptoms and hope by level of peer specialist engagement, with controls for relevant demographic factors, several baseline mental health and substance abuse measures, and service use during the study.
Results:
Logistic regression models showed that veterans with higher peer specialist engagement were more likely than those in a control group to show reliable positive change in psychiatric symptoms but not in hope. Compared with the control group, those with lower peer specialist engagement did not show positive change on either measure. White veterans were less likely than those from minority groups to exhibit positive change in psychiatric symptoms.
Conclusions:
Results suggest that peer specialists can benefit those with mental illnesses and substance use disorders who engage in more frequent interactions. Studies are needed to further assess the circumstances under which peer specialists can be effective.
Peer specialists—individuals with mental illness and substance use disorders who are trained to use their experiences to help others with similar disorders—have been used with increasing frequency in mental health services, but research evaluating their impact is mixed (1,2). Although several quasi-experimental and randomized trials of peer specialist services have demonstrated positive outcomes, including patient activation, self-efficacy, empowerment, hope, reduced symptom severity, and improved quality of life, other trials and meta-analyses have not yielded positive results (3–5). One design issue influencing the results of these trials is the assumption that peer specialist services benefit all participants in the same way. This assumption is consistent with intent-to-treat analyses, when all participants from an intervention group are compared with all participants of a control group, regardless of implementation variations.
However, the use of random assignment and intent-to-treat analyses could be considered an uneasy fit for peer specialist research, because peer specialists provide a unique type of support. Two theories underlying the work of peer specialists are social learning and social comparison theories (6,7), which emphasize the importance of the model (the peer specialist) being attended to and valued by the observer (the client). If a client does not, or is not able to, value and attend to the model, then the effectiveness of the peer specialist can be diminished for that client, compared with other clients who are more able to value the model and compared with other providers who do not rely as heavily on modeling and comparison (for example, psychiatrists). Thus the fit between client and peer specialist is uniquely important, compared with other treatment providers. Because this fit can vary, it is possible that within the samples of those receiving peer specialist services across the previous null-result trials, there are consistent subgroups of individuals who did benefit from those services.
As recommended in a peer specialist research agenda published recently in Psychiatric Services (8), more research is needed to assess not only whether peer specialists have impact at all but also when and for whom they have measurable impact. In this study, we tested whether there was a subgroup of individuals assigned to receive peer specialist services who exhibited reliably significant improvements on measures of symptoms of mental illness or hope (consistent with recovery) and whether receipt of more peer specialist services was associated with membership in that subgroup.
The AMPS Study: Administering MISSION-Vet Using Peer Support
The study reported here was a secondary analysis within a larger study of a two-site randomized trial. The larger study assessed a modified version of an evidence-based treatment for veterans with dual diagnoses of mental and substance use disorders, called MISSION-Vet (Maintaining Independence and Sobriety through Systems Integration, Outreach, and Networking–Veterans Edition) (9,10). MISSION-Vet was originally designed to be led by case managers with assistance from peer specialists. AMPS (Administering MISSION-Vet using Peer Support) used a modified version in which treatment is delivered solely by full-time peer specialists at each site. Peer specialists worked with veterans using the AMPS consumer workbook to provide 20 structured psychoeducation sessions and 20 unstructured visits, over approximately one year. The workbook contains discussion topics and worksheets, such as developing recovery goals, drawn from the MISSION-Vet materials, for peer specialists and veterans to work on together. Peer specialists at two northeastern VA medical centers were hired to provide this intervention to veterans currently enrolled in HUD-VASH (U.S. Department of Housing and Urban Development–Veteran Affairs [VA] Supportive Housing), a VA program that provides subsidized housing and ongoing case management support to veterans who are homeless. Large caseloads and active mental or substance use issues common among HUD-VASH–enrolled veterans (11–13) often make it difficult for case managers alone to address these disorders, even after the veterans obtain housing. The study randomly assigned veterans to standard HUD-VASH services or to HUD-VASH plus peer specialist services. The AMPS intent-to-treat analyses did not identify any effects of treatment assignment on measures of substance use and mental health (Ellison M, unpublished manuscript, 2018).
This Study
This study used the Reliable Change Index (RCI) (12) to attempt to determine whether there was a subgroup of veterans within the larger AMPS trial who may have improved on outcomes. As presented in several previous studies, the RCI is typically used to determine whether the amount of change among persons with mental illnesses who are exposed to a treatment is statistically reliable (14,15). The initial RCI formula subtracts the posttest score on an outcome measure from the baseline score and then divides by the standard error of the differences. The updated version of the RCI used here adjusts the pre-post difference score for regression to the mean because of measure unreliability (for RCI formula details, see 15).
The RCI formula was applied to a baseline-to-posttest measure of symptoms of mental illness (Behavior and Symptom Identification Scale [BASIS]) (16) and a measure of hope (Herth Hope Index [HHI]) (17) because of their documented high reliability and because previous studies have shown impact on these outcomes (18–27). The variable of interest that was assessed for predicting positive change in the intervention group was the extent of the relationship between the peer specialists and individuals assigned to work with them. Social comparison and social learning theories mentioned above and other theories about peer specialist effectiveness—for example, social support theories, experiential knowledge, and the helper-therapy principle (7,28–31)—all assert the importance of a strong relationship. Although multiple qualitative studies (including qualitative data from the larger study [32]) have supported that peer specialists are highly valued by the individuals with whom they work (33,34), significant variation can occur, as observed in the AMPS study (35) and in other studies of peer specialists working in homelessness services (36). Such findings suggest that veterans might not always be willing or able to engage with peer specialist services when such services are offered. These theories suggest the hypothesis that among veterans assigned to receive peer specialist services, those with greater engagement would be more likely to experience reliable improvement than those with less engagement.
There are few “as treated” analyses of the effectiveness of peer specialists, and the results are unclear. In one randomized trial whose intent-to-treat analyses showed no differences between peer specialist, community volunteer, and usual care groups (37), improvement was associated with having at least one contact with a community volunteer but zero contact with the assigned peer specialist. In another randomized trial (23), the as-treated analysis (at least one peer specialist contact) showed far fewer significantly improved alcohol and drug use outcomes but a few more physical health and functioning outcomes, compared with the intent-to-treat analysis.
This study built on this work and adjusted for sociodemographic and health-related variables (drug and alcohol use history, time homeless, and baseline mental health problems) that may influence the likelihood of positively changing when working with peer specialists. In addition, VA service utilization during the study that may have affected psychiatric symptoms and hope (that is, use of homelessness, substance use, and mental health services) were controlled for in all models to more comprehensively explore the relationship between peer specialist engagement and positive change.
Methods
Measures and Data Collection
The study took place between December 2012 and May 2017. All veterans provided informed consent in the AMPS study; they had already been enrolled in HUD-VASH and had a dual diagnosis (N=166). Eligibility criteria included active or recent history of substance abuse or dependence and a diagnosis of a mental disorder identified through VA records or as clinically judged by a case manager. Participants in the study reported here included only those who had completed study interviews before and after the approximate one-year treatment period (N=140) so that reliable change could be calculated. Those who completed the study did not differ from noncompleters on any demographic or other baseline covariates. All veterans were assessed with the following self-report measures: BASIS (38), the Addiction Severity Index (ASI) (39), the HHI (17), and the Temple University Community Participation Measure (40). Data on use of VA homelessness, substance use, mental health, and peer specialist services were extracted from the VA Common Data Warehouse (CDW) (41).
Dependent Variables
As described above, the HHI and BASIS were used to categorize all participants as having either exhibited positive change or no or negative change. The RCI index is sensitive to measurement reliability (internal consistency and Cronbach’s alpha). Thus, to maximize the identification of positive and negative changers, we wanted to use measures with well-documented high reliability in similar populations. Of the measures used in the AMPS study, only the HHI and BASIS had sufficiently high reliability. In addition, these measures address aspects of recovery and symptoms, both of which have shown improvement in previous studies of peer specialists. For example, we chose the HHI because maintaining hope that one can improve is viewed as an important part of recovery (42), and multiple studies have shown that peer specialists can build hope (18,21,24,26,27). The HHI is a 12-item index adapted from the 30-item Herth Hope Scale. It has been shown to be reliable (α=.97) and valid, correlating with the Existential Well-Being Scale (43) (r=.84) and the Hopelessness Scale (44) (r=–.73) in a sample of ill adults (17).
We chose the 24-item BASIS because of its widespread use as a comprehensive measure of psychiatric symptoms and because multiple studies have shown that peer specialists can improve such symptoms (19,20,22,23,25,26). It consists of an overall summary score that includes six domains: depression-functioning, interpersonal relationships, self-harm, emotional lability, psychotic symptoms, and substance abuse. In a national sample of more than 3,200 recipients of mental health or substance abuse services, the BASIS summary score had high reliability (α=.90) (38). We applied the RCI equation (15) to compute whether each participant achieved reliable positive change (coded as 1) or negative or no reliable change (coded as 0) on the HHI and BASIS, and the resulting dichotomous variables were used as the dependent variable in logistic regression analyses (see below).
Independent Variables
To test engagement with peer specialist services and still be able to make comparisons to participants in the control group, the engagement variable (number of visits during the study) was dichotomized on the basis of a median split, creating a “low peer specialist engager” group (<12 contacts) and a “high peer specialist engager” group (≥12 contacts). Two dummy variables were created to compare each engagement group with participants in the control group, with one variable representing control group versus low engagers and the second representing control group versus high engagers. We did this because the distribution of peer specialist services was mostly bimodal—that is, some intervention participants saw their assigned peer specialists infrequently while others met with their peer specialist often.
All covariates, except service utilization, were collected at baseline. Participants were asked their age and race (selected among seven different racial categories and recoded into white and nonwhite because of lower frequencies in nonwhite categories). Lifetime homelessness categories (5) were recoded to one year or less of lifetime homelessness and more than one year on the basis of data distributions. Data on drug and alcohol use and psychological problems were collected with the ASI, a widely used assessment with strong concurrent and discriminant validity among persons with substance use disorders and psychiatric disorders (39). Participants self-reported years of lifetime alcohol intoxication and years of use for ten illicit or inappropriately used substances—heroin, methadone, other opiates, barbiturates, sedatives, cocaine, amphetamines cannabis, hallucinogens, and inhalants—averaged across the ten variables. Psychological and emotional problems were the number of days participants reported that they had experienced such problems in the past month.
Last, additional variables included as covariates were study site (site 1 or site 2) and use of homelessness, substance use, and mental health services during the study; data on service use were obtained from the VA CDW. Service use contacts were divided by total months enrolled to adjust for differing enrollment lengths across participants (mean±SD=11.49±3.24 months).
Data Analysis
Our primary analyses were conducted to predict positive change on the HHI and BASIS by using logistic regression. Prior to running logistic regression models, we tested for significant differences between participants in the control group, low peer specialist engagers, and high peer specialist engagers on various demographic and other baseline variables (Table 1). Bivariate correlations were computed to examine the interrelatedness of all study variables and to help assess for multicollinearity between predictors before the regression analyses were conducted. No predictors were correlated at ≥.90, indicating a low likelihood of collinearity problems (45). Moreover, tolerance and variance inflation factors for each predictor were computed, and all were in acceptable ranges (tolerance values all >.1, and variance inflation factors all <2.5), further indicating low probability of collinearity (46).
| High engagement(N=38) | Low engagement(N=37) | Control(N=65) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Variable | N | % | N | % | N | % | Test statistic | df | p |
| Male | 36 | 95 | 35 | 95 | 58 | 89 | .55a | ||
| White | 18 | 47 | 17 | 46 | 39 | 60 | χ2=2.50 | 2 | .29 |
| Site 1 | 24 | 63 | 12 | 32 | 31 | 48 | χ2=7.09 | 2 | .03 |
| Lifetime homelessness >1 year | 21 | 55 | 24 | 65 | 49 | 75 | χ2=4.52 | 2 | .10 |
| Herth Hope Index, positive change | 10 | 27 | 4 | 11 | 12 | 19 | χ2=3.01 | 2 | .22 |
| Behavior and Symptom Identification Scale, positive change | 15 | 41 | 9 | 24 | 7 | 11 | χ2=11.05 | 2 | .004 |
| Age (M±SD years) | 54.08±6.86 | 54.00±8.97 | 52.51±8.81 | χ2=2.41b | 2 | .30 | |||
| Lifetime alcohol intoxication (M±SD years) | 16.97±15.48 | 19.16±16.63 | 13.95±13.43 | F=1.52 | 2, 137 | .22 | |||
| Lifetime drug use (M±SD years) | 3.66±4.43 | 3.34±3.65 | 3.13±3.09 | χ2=.009b | 2 | .995 | |||
| Days with psychiatric problems in past 30 | 9.61±12.19 | 12.95±12.74 | 9.94±11.81 | F=.90 | 2, 135 | .41 | |||
| Peer specialist engagementc | 25.55±8.82 | 5.19±3.89 | — | t=–13.49 | 111 | <.001 | |||
| Mental health service utilizationd | .98±1.75 | 1.32±3.35 | .65±.80 | χ2=.20b | 2 | .91 | |||
| Homelessness service utilizationd | 2.84±1.74 | 2.18±1.36 | 2.44±1.47 | χ2=3.13b | 2 | .21 | |||
| Substance use service utilizationd | 1.92±3.45 | 1.22±1.38 | 1.70±3.88 | χ2=2.00b | 2 | .37 | |||
TABLE 1. Baseline and service use variables for 140 veterans with a dual diagnosis, by study group
We performed two separate binary logistic regression models assessing the likelihood of positive change on the HHI and BASIS (0, no or negative change; 1, positive change). As stated above, each model included two dummy variables testing peer specialist engagement: control group versus low engagement and control group versus high engagement. Both models adjusted for the following covariates: age; race; study site; history of lifetime homelessness; lifetime alcohol intoxication; lifetime average drug intoxication; frequency of past-30-day emotional or psychological problems; and use of homelessness, substance use, and mental health services during the study. Odds ratios (ORs) and their 95% confidence intervals were computed to determine effect sizes and statistical significance. Analyses were performed with IBM SPSS Statistics 24 and SAS, version 9.4.
Results
Sample Characteristics
Of the 140 participants, 65 had been randomly assigned to the control group and 75 to the peer specialist intervention groups at baseline. The mean age of participants was 53.33±8.35 (range, 23–67), and 92% (N=129) identified as male. Participants self-reported as being from the following racial groups: white/Caucasian, 53% (N=74); African American/black, 38% (N=54); multiracial, 6% (N=8); other, 2% (N=3); and American Indian/Alaska Native, 1% (N=1).
Table 1 presents data on the study variables for the high and low engagement groups and the control group. The three groups differed significantly by study site, with site 1 having more high engagers than site 2. In addition, the proportion of participants with a positive change on the BASIS was significantly larger in the high engagement group than in the low engagement group.
Bivariate Correlations
Bivariate correlations between all study variables are presented in Table 2. High peer specialist engagement was significantly related to being in the positive change group on the BASIS (r=.26, p<.01), and low peer specialist engagement was not. The level of peer specialist engagement differed by site but was not significantly correlated with any other variable. Being in the positive change group on the HHI was positively correlated with lifetime alcohol use frequency (r=.18, p<.05), whereas being in the positive change group on the BASIS was correlated with identifying as a being from a minority racial or ethnic group (r=–.19, p<.05) and with less time homeless (r=–.18, p<.05). Participants in the positive change group on the BASIS were more likely to also be in the positive change group on the HHI (r=.28, p<.01).
| Variable | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1. Age | ||||||||||||||
| 2. White | –.14 | |||||||||||||
| 3. Site 1 | .09 | –.53** | ||||||||||||
| 4. Lifetime homelessness >1 year | .11 | .15 | –.38** | |||||||||||
| 5. Lifetime alcohol intoxication (years) | .24** | .14 | –.13 | .14 | ||||||||||
| 6. Lifetime drug use (years) | .10 | .14 | –.01 | .10 | .35** | |||||||||
| 7. Days with psychiatric problems in past 30 | –.14 | .12 | –.10 | –.05 | .09 | .02 | ||||||||
| 8. Mental health service use | .12 | .23** | –.16 | –.04 | –.04 | –.05 | .10 | |||||||
| 9. Homelessness service use | .04 | –.04 | .19* | .03 | .06 | .03 | –.03 | –.12 | ||||||
| 10. Substance use service use | –.15 | .14 | –.11 | .18* | .02 | .13 | –.05 | –.03 | .08 | |||||
| 11. High peer specialist engagement | .06 | –.07 | .19* | –.16 | .03 | .06 | –.05 | .02 | .14 | .05 | ||||
| 12. No or low peer specialist engagement | .05 | –.08 | –.19* | –.03 | .12 | .00 | .11 | .12 | –.12 | –.08 | –.37** | |||
| 13. HHI positive changeb | –.03 | .13 | –.03 | .04 | .18* | .12 | .15 | –.01 | –.07 | –.01 | .13 | –.12 | ||
| 14. BASIS positive changec | .04 | –.19* | .06 | –.18* | .05 | –.04 | .14 | .17 | –.07 | –.15 | .26** | .02 | .28** | |
TABLE 2. Bivariate correlations between study variablesa
Logistic Regressions Predicting Positive Change in the HHI and BASIS
Table 3 presents the logistic regression models for predicting the likelihood of being in the positive change group on the HHI (model 1) and BASIS (model 2). Peer specialist engagement did not predict the likelihood of being in the positive change group on the HHI. Only one significant covariate emerged in model 1: experiencing more years of alcohol intoxication over the course of a veteran’s lifetime was associated with a greater likelihood of being in the positive change group on the HHI (OR=1.04, p=.05). Results for model 2 indicated that high peer specialist engagers were five times more likely than control group participants to be in the positive change group on the BASIS (OR=5.51, p<.01). In addition, veterans identifying as white were less likely than those in the other racial-ethnic groups to be in the positive change group on the BASIS.
| HHI positive change | BASIS positive change | |||||
|---|---|---|---|---|---|---|
| Variable | OR | 95% CI | p | OR | 95% CI | p |
| Age | .98 | .93–1.04 | .57 | .99 | .93–1.05 | .69 |
| White (reference: nonwhite) | 2.09 | .61–7.21 | .24 | .13 | .03–.55 | .006 |
| Site 1 (reference: site 2) | 1.44 | .39–5.35 | .59 | .33 | .08–1.33 | .12 |
| Lifetime homelessness >1 year (reference: ≤1 year) | 1.41 | .41–4.77 | .58 | .45 | .14–1.41 | .17 |
| Lifetime alcohol intoxication (years) | 1.04 | 1.00–1.07 | .05 | 1.02 | .98–1.06 | .30 |
| Lifetime drug use (years) | 1.01 | .88–1.15 | .93 | .95 | .79–1.13 | .56 |
| Days with psychiatric problems in past 30 | 1.04 | 1.00–1.08 | .06 | 1.03 | .99–1.07 | .11 |
| Mental health service use | .87 | .59–1.30 | .51 | 1.27 | .96–1.69 | .09 |
| Homelessness service use | .79 | .55–1.14 | .22 | .85 | .59–1.22 | .36 |
| Substance use service use | .95 | .81–1.11 | .54 | .81 | .60–1.09 | .17 |
| High peer specialist engagement | 1.98 | .65–6.07 | .23 | 5.51 | 1.63–18.61 | .006 |
| No or low peer specialist engagement | .48 | .12–1.92 | .30 | 1.42 | .41–4.96 | .58 |
TABLE 3. Likelihood of positive change on the Herth Hope Index (HHI) and the Behavior and Symptom Identification Scale (BASIS) among veteransa
Discussion and Conclusions
The findings suggest that there was a subgroup of individuals assigned to work with peer specialists who exhibited reliable positive change after about one year of peer specialist services, even when results of the intent-to-treat analysis did not indicate significant improvement. Over 40% of participants with a high level of engagement with their peer specialist showed reliable positive change in their symptoms (BASIS), compared with only 24% of those with a low level of engagement and 11% of those in the control group. The logistic regression model showed the same result, even after the analysis controlled for several relevant covariates. Having a high level of peer specialist engagement was the only variable associated with a positive change in symptoms, whereas being white was associated with a lower likelihood of having a positive change in symptoms. Other studies have shown that peer specialists can be helpful particularly to those who are African American (47). However, it is possible that persons of color were more likely to exhibit positive changes in symptoms because traditionally these groups have had difficult accessing services, and the HUD-VASH program provided greater access to services regardless of the presence of a peer specialist. The level of peer specialist engagement was not related to reliable positive change on the HHI, possibly because receiving a housing voucher had already led to improved feelings of hope.
These findings suggest that the use of peer specialist services may be a matter of fit between the client and the peer specialist. This point was suggested in an early review of peer support groups (48), which documented that initial participation often fell off but that significant benefits accrued for those whose group attendance was higher. It appears that the same may be true for high versus low users of individual peer specialist services. The beneficial effect of greater contact with peer specialists is consistent with qualitative interviews from this study (32) and with other studies of veterans receiving peer specialist services (49). In particular, those in this study who were high peer specialist engagers expressed strongly that they benefited from their relationship. These findings could have important policy implications for the use of peer specialists in clinical settings. Although peer specialists are sometimes used in roles that limit relationship building (for example, greeter or driver), the findings of this study suggest that it may be beneficial to allow peer specialists to develop strong, longer-term relationships with those who want that type of support.
Certain limitations should be noted. These results are from a small sample in two VA medical centers. In addition, the use of high and low peer specialist engagement groups in regression analyses did not allow for a determination of causality. Larger-scale studies that can randomly assign individuals to lower and higher amounts of peer specialist services are needed to further explore these results. Qualitative studies that explore reasons for lack of engagement would also be useful. The use of a median split to categorize the level of peer specialist services to assess the impact of engagement is a somewhat rudimentary approach. However, the analysis of peer specialist engagement is in its infancy. For example, the two studies mentioned above (23,37) that also analyzed peer specialist engagement used no engagement versus some. Median split—hypothesizing that more contacts than the median would be associated with positive change—is a small advance over that approach. Finally, this study did not control for peer specialist characteristics that may have influenced the relationship between peer engagement frequency and positive change. We hope that this study will stimulate future research (with larger sample sizes) that can begin to address the issue of the exact amount of peer specialist services that are needed to achieve positive outcomes.
Despite these limitations, this study showed that veterans with mental illnesses and substance use disorders who engaged in more frequent interactions with peer specialists benefited. The type of subgroup analysis conducted in this study adds to nascent literature evaluating the impact of variable engagement as envisioned by the peer specialist research agenda mentioned above (8). More studies are needed to investigate the impact of various patient and peer specialist factors on outcomes, which could shed light on the circumstances under which peer specialists can be most effective.
1 : Peer-delivered services: current trends and innovations. Psychiatric Rehabilitation Journal 39:193–196, 2016Crossref, Medline, Google Scholar
2 : Peer support services in the behavioral healthcare workforce: state of the field. Psychiatric Rehabilitation Journal 39:197–203, 2016Crossref, Medline, Google Scholar
3 : Peer support services for individuals with serious mental illnesses: assessing the evidence. Psychiatric Services 65:429–441, 2014Link, Google Scholar
4 : A systematic review and meta-analysis of randomised controlled trials of peer support for people with severe mental illness. BMC Psychology 14:39, 2014Crossref, Google Scholar
5 : Consumer-providers of care for adult clients of statutory mental health services. Cochrane Database of Systematic Reviews (3):CD004807, 2013Google Scholar
6 : Peers and peer-led interventions for people with schizophrenia. Psychiatric Clinics of North America 35:699–715, 2012Crossref, Medline, Google Scholar
7 : Social Learning Theory. New York, General Learning Press, 1977Google Scholar
8 : Establishing a research agenda for understanding the role and impact of mental health peer specialists. Psychiatric Services 68:955–957, 2017Link, Google Scholar
9 : A brief treatment engagement intervention for individuals with co-occurring mental illness and substance use disorders: results of a randomized clinical trial. Community Mental Health Journal 48:127–132, 2012Crossref, Medline, Google Scholar
10 : A wraparound treatment engagement intervention for homeless veterans with co-occurring disorders. Psychological Services 10:161–167, 2013Crossref, Medline, Google Scholar
11 : Using Getting To Outcomes to facilitate the use of an evidence-based practice in VA homeless programs: a cluster-randomized trial of an implementation support strategy. Implementation Science 12:34, 2017Crossref, Medline, Google Scholar
12 : The 2015 Annual Homeless Assessment Report (AHAR) to Congress. Washington, DC, US Department of Housing and Urban Development, 2015Google Scholar
13 : Housing chronically homeless veterans: evaluating the efficacy of a Housing First approach to HUD‐VASH. Journal of Community Psychology 41:505–514, 2013Crossref, Google Scholar
14 : Measuring clinically meaningful change following mental health treatment. Journal of Behavioral Health Services and Research 34:272–289, 2007Crossref, Medline, Google Scholar
15 : Behavior and Symptom Identification Scale 32: sensitivity to change over time. Journal of Behavioral Health Services and Research 32:341–346, 2005Crossref, Medline, Google Scholar
16 : BASIS-32 and the Revised Behavior and Symptom Identification Scale (BASIS-R); in The Use of Psychological Testing for Treatment Planning and Outcome Assessment, 3rd ed, vol. 3. Edited by Maruish M. Mahwah, NJ, Erlbaum, 2004Google Scholar
17 : Abbreviated instrument to measure hope: development and psychometric evaluation. Journal of Advanced Nursing 17:1251–1259, 1992Crossref, Medline, Google Scholar
18 : A randomized controlled trial of the effectiveness of a modified recovery workbook program: preliminary findings. Psychiatric Services 60:491–497, 2009Link, Google Scholar
19 : A user-developed, user run recovery programme for people with severe mental illness: a randomised control trial. Psychosis 8:287–300, 2016Crossref, Google Scholar
20 : Results of a randomized controlled trial of mental illness self-management using Wellness Recovery Action Planning. Schizophrenia Bulletin 38:881–891, 2012Crossref, Medline, Google Scholar
21 : Randomized controlled trial of peer-led recovery education using Building Recovery of Individual Dreams and Goals through Education and Support (BRIDGES). Schizophrenia Research 136:36–42, 2012Crossref, Medline, Google Scholar
22 : The Buried in Treasures Workshop: waitlist control trial of facilitated support groups for hoarding. Behaviour Research and Therapy 50:661–667, 2012Crossref, Medline, Google Scholar
23 : Outcomes of a peer mentor intervention for persons with recurrent psychiatric hospitalization. Psychiatric Services 69:760–767, 2018Link, Google Scholar
24 : Influence of single peer interventions on the recovery attitude of persons with a psychiatric disability. Scandinavian Journal of Caring Sciences 26:755–760, 2012Crossref, Medline, Google Scholar
25 : Efficacy of dual focus mutual aid for persons with mental illness and substance misuse. Drug and Alcohol Dependence 135:78–87, 2014Crossref, Medline, Google Scholar
26 : Outcomes from consumer-operated and community mental health services: a randomized controlled trial. Psychiatric Services 62:915–921, 2011Link, Google Scholar
27 : Effects of a peer-run course on recovery from serious mental illness: a randomized controlled trial. Psychiatric Services 63:54–60, 2012Link, Google Scholar
28 : The “helper-therapy” principle. Social Work 10:27–32, 1965Google Scholar
29 : The client as helper: a means to promote psychological growth. Counseling Psychologist 43:58–64, 1974Crossref, Google Scholar
30 : Understanding Self-Help and Mutual Aid: Experiential Learning in the Commons. New Brunswick, NJ, Rutgers University Press, 1999Google Scholar
31 : A theory of social comparison processes. Human Relations 7:117–140, 1954Crossref, Google Scholar
32 : Peer specialists: exploring the influence of program structure on their emerging role. Psychological Services, in pressGoogle Scholar
33 : Clarifying the role of the mental health peer specialist in Massachusetts, USA: insights from peer specialists, supervisors and clients. Health and Social Care in the Community 22:104–112, 2014Crossref, Medline, Google Scholar
34 : Individual peer support: a qualitative study of mechanisms of its effectiveness. Community Mental Health Journal 51:445–452, 2015Crossref, Medline, Google Scholar
35 : Patterns and predictors of engagement in peer support among homeless veterans with mental health conditions and substance use histories. Psychiatric Rehabilitation Journal 39:266–273, 2016Crossref, Medline, Google Scholar
36 : Homeless veterans’ use of peer mentors and effects on costs and utilization in VA clinics. Psychiatric Services 68:628–631, 2017Link, Google Scholar
37 : Supported socialization for people with psychiatric disabilities: lessons from a randomized controlled trial. Journal of Community Psychology 32:453–477, 2004Crossref, Google Scholar
38 : The Revised Behavior and Symptom Identification Scale (BASIS-R): reliability and validity. Medical Care 42:1230–1241, 2004Crossref, Medline, Google Scholar
39 : The fifth edition of the Addiction Severity Index. Journal of Substance Abuse Treatment 9:199–213, 1992Crossref, Medline, Google Scholar
40 : Measuring community participation of adults with psychiatric disabilities: reliability of two modes of data collection. Rehabilitation Psychology 59:211–219, 2014Crossref, Medline, Google Scholar
41 : Insights from advanced analytics at the Veterans Health Administration. Health Affairs 33:1203–1211, 2014Crossref, Google Scholar
42 : An exploratory analysis of correlates of recovery. Psychiatric Services 55:540–547, 2004Link, Google Scholar
43 : Loneliness, spiritual well-being and the quality of life; in Loneliness: A Sourcebook of Current Theory, Research and Therapy. Edited by Peplau L, Perlman D. New York, Wiley, 1982Google Scholar
44 : The measurement of pessimism: the Hopelessness Scale. Journal of Consulting and Clinical Psychology 42:861–865, 1974Crossref, Medline, Google Scholar
45 : Experimental Designs Using ANOVA. Belmont, CA, Duxbury, 2007Google Scholar
46 : Collinearity diagnostics of binary logistic regression model. Journal of Interdisciplinary Mathematics 13:253–267, 2013Crossref, Google Scholar
47 : Using peer navigators to address the integrated health care needs of homeless African Americans with serious mental illness. Psychiatric Services 68:264–270, 2017Link, Google Scholar
48 : Peer support among individuals with severe mental illness: a review of the evidence. Clinical Psychology 6:165–187, 1999Google Scholar
49 : Implementation of consumer providers into mental health intensive case management teams. Journal of Behavioral Health Services and Research 42:100–108, 2015Crossref, Medline, Google Scholar
