The Consequences of Managing to Data: The VA Example
It’s not secret that many veterans experience trauma related to their service and, at times, their return home after active duty. Recently, the Departments of Veteran’s Affairs (VA) changed its focus to increase counselor productivity due to increases in demand. Generally, an increase in productivity would be a great thing to help veterans in need get better access to mental health care. However, in this case managing to the data has had unexpected consequences.
According to this story by NPR, the VA has set 25 visits a week per counselor as a “fully successful” metric. It also set the approximate time for those visit at 17 hours, with additional hours beyond that gaining an “excellent” rating.
The issue with this metric is that right away it does not pass the sniff test. If you’ve ever seen a therapist, grief counselor or even many types of social worker, you’d know the typical appointment is set for an hour. Particularly difficult cases may also start out with longer sessions of an hour and a half or two hours. If you’ve ever known a therapist, you also know that the appointment doesn’t end when the patient leaves. Typically the counselor needs to log notes about the discussion and, in the case of those that can prescribe medicine, prescriptions may need to be written and faxed to the appropriate pharmacies. A small office may not have a scheduler, so the counselor may need to wear that hat as well. These processes add at least 30 minutes to each appointment.
This means the VA’s estimate is already 20.5 hours short of the approximately 37.5 hours it would take to visit with 25 patients. In fact, the VA itself calculated the average visit time at 77 minutes, which would add up to 32 hours.
An additional factor is that certain locations where counselors are located naturally have lower traffic, but the standards set for the counseling offices are uniform. One counselor interviewed by NPR has chosen to retire rather than meet the standards set by the VA and suggests, “…there needs to be more hiring at higher echelons of people in managerial and leadership positions that have clinical backgrounds so they can make common-sense clinical decisions.”
This case is a perfect example where managing to a set of data seems like a good idea, but only if the information is presented meaningfully to leaders who are informed on the day-to-day operations of the organization.
Accuracy & Critical Consumption
The fact that the standard set exceeded even a layperson’s operational understanding by 150% suggests that the decision makers in this process received inaccurate data about visit times at some point during the process. This could be because of a display error or confusing graphic, or simply a typo in a report that was recirculated.
Everyone makes mistakes, but building peer review into your analytics process helps reduce these kinds of errors. It also helps to have someone from another department to review graphics where possible, since graphical literacy cannot be assumed across the organization.
That said, it is still the responsibility of those consuming the data to do so with a critical eye. At some point, someone in leadership should notice that the target hours didn’t match the internal time estimate. You don’t need experience in the field or a mathematics degree to notice that 25 x 1.2 does not equal 17. A good organization will have two way communication between analysts and leadership to clarify information in cases if conflicting information.
Reducing for the Sake of Reduction
There’s also a chance that all the information was presented accurately and correctly, but decision makers felt that 77 minutes was too long for appointment times for any number of reasons that weren’t communicated to the field. It’s possible that this idea might be justified, but the process for doing so would involved getting input from providers and patients, completing test cases and understanding whether the appointment time is meant to be measured alone or is supposed to include clerical duties.
What’s more likely is that leadership decided to set aggressive targets to increase patient visits without knowing how damaging reducing the time per visit would be to quality. A similar scenario are ever-increasing sales goals that never reach a maximum and discourages employees. If the process isn’t changing and the resources aren’t changing, it’s very difficult to make large changes in metrics, particularly when the reasoning isn’t fully communicated with providers.
Good leaders will spend the time to understand what’s behind the numbers, rather than just assuming aggressive targets can be met with enough effort.
Lack of Field Experience
The quote cited above indicates that many of the decision makers don’t have clinical experience, or at least are perceived that way by the general population of counselors in the field. Since this case involves government bureaucracy, it’s not altogether surprising that lack of experience is present in leadership since positions are often appointed from politicians rather than clinicians. That said, it’s not an impossible problem to fix. Regular two-way communication with just a few selected members of the field can make a big difference in the general understanding of statistical information. Patient responses can also be synthesized into a narrative to explain the unique aspects of treating veterans for new leaders in the VA. Additionally, not every senior leader needs field experience, but they should defer to those that do have field experience to speak for the employee during the decision-making process.
Data alone might not be enough to understand the full picture to set appropriate targets for clinicians at the VA or in any large organization. Leaders should avoid relying too much on raw data without understanding the daily dynamics of the organization. Simple visualizations and cross-departmental referencing can help leaders avoid the problems associated with the VA’s new targets. Talk to us today about how to set up meaningful dashboards to help you make better decisions for your business and employees.