Envisioning Transformation in VA Mental Health Services Through Collaborative Site Visits

The Veterans Health Administration (VHA) Office of Mental Health Operations (OMHO) was founded to monitor implementation of mental health policy and to partner with regional and field-based staff in collaborative quality improvement (QI) processes to advance the delivery of mental health and substance use disorder services. Rather than employ the standard strategies for implementing new policies—namely, mandating change and training staff—OMHO moved beyond these strategies, adopting methods from implementation science to guide systemwide QI to improve access to services and quality of care.

As part of this QI initiative, each VHA facility received a comprehensive site visit to review policy implementation across all areas of mental health. The site visits were intended to help VHA achieve its vision of implementing high-quality mental health services. This column describes lessons learned during the development of the site visit QI initiative as well as the goals accomplished and the challenges faced during the process. Other health care systems may wish to consider these lessons as they develop their own QI initiatives.

The Site Visit Process

From October 2011 through September 2012, OMHO conducted comprehensive, consultative, baseline mental health site visits at all 140 VHA health care systems. The visits were led by OMHO’s newly created technical assistance (TA) team in conjunction with field-based site visitors. Each health care system received a second site visit between October 2012 and September 2015. Visits involved interviews with facility leadership, mental health leadership and program managers, frontline staff, veterans receiving services and family members and other individuals who provided the veterans with support, and community partners. These visits were focused on providing in-depth reviews of policy implementation across all mental health areas. OMHO followed all site visits with a facility-specific, consultative strategic action planning process to facilitate addressing areas of improvement needed as part of the QI process.

Lessons Learned

Mandates alone do not lead to QI changes.

In 2008, the VHA published the Uniform Mental Health Services Handbook (UMHSH) (1), which provides a groundbreaking policy that outlines minimum mental health services requirements across the VHA. However, a 2011 facility self-report survey measuring the presence of mental health services (2) found that only 69% of VHA medical centers had implemented at least 95% of the required elements. Policy mandates had not resulted in desired systemwide change. Development of national standards of care and training initiatives were insufficient in ensuring local implementation of change management processes.

The challenge was clear—providing the range and quality of mental health services envisioned by the VHA required that OMHO engage the field in continuous dialogue and solicit its participation in the change process. It was critical to emphasize engagement by all stakeholders in all phases of implementation (3). Thus OMHO created site visits to enhance policy implementation and promote improved QI processes. The visits provided an opportunity to assess each site and obtain important data about facilitators of and barriers to the delivery of high-quality services, while informing leadership about needed support and modeling a collaborative, consultative QI approach.

Partnerships are key to systemwide change.

Collaboration across all levels of a health care system is needed to improve service delivery (4). The site visit process was designed around a foundation of partnership, including partnerships within the newly formed OMHO (3). [A table listing partners’ tasks is available as an online supplement to this article.] To be successful, the team guiding large, systemwide change must include individuals with credibility, content expertise, and the ability to work with and motivate others (3). Within OMHO, the clinical and administrative experience of the TA team was brought together with the program evaluation and survey development skills of the Program Evaluation Center team. A site visit protocol was developed via interteam meetings to allow for cross-fertilization of ideas. Together these teams created a standardized protocol that included administrative data for each facility as a tool to highlight areas requiring more in-depth individualized analysis by site (5). This process became the prototype for informing upper management at the national and facility level of concerns identified through administrative data and from discussions with key stakeholders across the system.

Policy, research, and field-based stakeholders should be engaged across the system.

The OMHO guiding leadership team identified the need to actively engage stakeholders in the change process (3). It obtained input from national policy partners, researchers, regional and facility mental health leadership, mental health experts in the field, and veterans and their family members. Ultimately, this improved the site visit process and its implementation success (6). Leaders of the OMHO team proactively communicated the urgent need for change, reflecting upon the gap between the current and desired implementation of mental health services. Partnerships with these stakeholders merged into a vested interest in the shared vision of site visit QI (7). Partners helped to improve the standardized site visit protocol, providing suggestions for additional content. Field and regional stakeholders were particularly central to ensuring that all processes would be viewed as logical and reasonable to the facility-level staff responsible for local change.

In order to maximize diversity in profession, expertise, and organizational position, site visit teams consisted of facility, regional, and national stakeholders. Thus facilities were visited by a team of peers, increasing credibility and acceptance of site visit findings. Furthermore, the site visitor teams themselves developed strong relationships around a shared purpose of providing consultation and enhancing services. Many former site visitors continue to serve as consultants for VHA, for example, serving on temporary leadership assignments to other sites and providing consultation.

Finally, stakeholders across system levels were actively involved in evaluating the site visit process. Multiple opportunities were available to provide constructive feedback, which was used to refine the site visit process. By actively seeking feedback from multiple stakeholders, the OMHO leadership team engaged the entire mental health care system in the effort, creating a platform for empowerment in the change process and for ongoing dialogue about QI (3).

Misunderstandings and lack of trust across the system must be addressed.

Prior to the initiation of the site visits, national leadership often heard complaints from local and regional leaders that national leaders did not understand the complexity of implementing policy. As is common in organizational hierarchies (6), there was distrust of national leaders and their vision for mental health services, with frequent misunderstandings of national mental health policies. By visiting facilities in person, OMHO leaders made themselves accessible to providers in the field and observed firsthand the policy implementation challenges that confronted VHA facilities. National and regional leaders strengthened their relationships by working together to address facility challenges. Relationships forged in these processes have developed into ongoing dialogue about methods to overcome system challenges and improve problem areas identified by sites.

OMHO site visit teams systematically identified strengths at each facility and frequently identified strong practices, recognizing that even sites with many challenges may have areas of strength. These were highlighted in site visit reports and in exit briefings with senior facility and Veterans Integrated Service Network leaders. This effort helped to build mutual respect and trust.

Administrative mental health data alone cannot assess full policy implementation.

OMHO developed a range of quantitative metrics from administrative data to monitor mental health policy implementation. Although administrative information provided a range of data to assess policy implementation, local interpretation of coding guidance and data entry procedures affected data quality, potentially diminishing their reliability (5,8). Another lesson learned was that administrative data without context about care processes could be misleading. Data outliers may signal services of truly poor quality but also may result from data entry error or idiosyncratic processes. Single data points cannot adequately describe how well sites implement policy (8). Administrative data are also limited by existing measurement systems. The site visits reinforced that not everything of interest can be measured. Although rare, safety issues requiring immediate attention were identified on site visits, as were concerns about work climate and morale that required intervention. Through collaborative site visits, metrics were placed in context and were used to understand the challenges facing VHA facilities as well as to identify strong practices.

Accomplishments of the Site Visit Process

Overarching goals.

The site visit process aimed to assess and enhance implementation of mental health policy, identify areas for QI, empower local QI processes, obtain background information, identify strong practices to share nationally, and provide a cadre of trained reviewers to support national efforts to achieve the larger goal of improving mental health care. These goals were accomplished by using implementation science strategies and focusing on partnerships. Through the site visit process, 69 strong practices were identified and highlighted in a national repository as a resource for dissemination. These strong practices served as a resource for sites working on QI in a stated area. The percentage of facilities that had implemented the UMHSH rose from 69% in fiscal year 2011 (prior to site visits) to 91% in fiscal year 2015 (following site visits). A September 2015 analysis found that sites had completed 72% of initial site visit recommendations.

Case illustration.

One notable example of process improvement was in the implementation of primary care–mental health integration (PCMHI). Integration of mental health services in primary care has been critical in reducing stigma in accessing services, decreasing wait time for services, and increasing treatment engagement for veterans reporting mental or behavioral health concerns for the first time. VA policy requires both colocated collaborative care and care management services at all medical centers and larger outpatient clinics (1). However, despite additions in PCMHI staffing, in 2011 only 37% of sites provided both required services and 18% of sites provided no PCMHI services.

The site visits provided critical information on the local challenges of implementation and an opportunity to connect local facilities with national resources and consultation support. With ongoing monitoring of local progress and connection to technical assistance, great strides were made. By 2015, 70% of sites had fully implemented PCMHI, nearly doubling the implementation, and only 8% of sites reported having no PCMHI services. These improvements laid the foundation for same-day access to mental health services in primary care and improved treatment engagement.

Challenges and Next Steps

Site visits provided information that was not revealed through administrative data, in some cases identifying issues that require an immediate response, such as concerns about access, patient flow, work climate, and safety. OMHO site visits also identified numerous strong practices and modeled changes in collaborative processes, resulting in improved mental health care.

Although the site visit process successfully improved policy implementation across the system, further improvements were needed after the visits ended. At some sites, establishing goals and improving partnerships were critical in promoting overall culture change. At other programs, particularly well-run mental health programs, the standardized site visit and action planning process were perceived as micromanaging. A stepped-level site visit intervention process, with more intensive oversight for lower-performing facilities, has now been developed. Facilities consistently performing well on measures of quality are visited less frequently, allowing room for focused assistance at sites needing greater support. This approach has been well received by facilities and senior leadership, given that it preserves costly resources and is responsive to field input.

Opportunities for consultation remain, in that facilities are able to request assistance on an as-needed basis. However, because many sites will no longer be visited, the elimination of routine visits may weaken the personal connections and trust so carefully built across the nation, causing a potential loss of rich local information obtained by visiting every system. It may be prudent to revisit this process again in several years to determine whether additional rounds of routine visits might boost the effects described above.

Implications for Other Health Care Systems

The complex VA mental health care system is dependent on a strong relationship infrastructure. The array of relationships between administrators, clinicians, veterans, community stakeholders, and evaluators is the foundation for true QI and culture change (9). OMHO processes have been strengthened by listening and responding to partner and stakeholder feedback when developing, implementing, and refining this nationwide QI process. As other health care systems consider adaptation of site visit processes for QI, paying attention to systemwide relationships will be key. For QI to succeed, it must be built on trust. Stakeholders must be assured that leaders not only have content expertise but also feel personal care and concern toward providers and patients (3).

In addition to developing trust and building relationships, individuals are encouraged to remember that metrics alone will not effectively evaluate a full system. Metrics are simply indicators of processes and outcomes that will need further evaluation with qualitative examination, best completed in partnership with local stakeholders. It is not uncommon to satisfy a metric with a process that does not create the desired improvement in the quality of care. Remote data viewed from a national “telescope” are insufficient for evaluation of the quality of mental health services. For true QI, providers at large systems must partner across each level to obtain a full view of the entire system so that shared strategic planning for QI can be conducted as one team.