What’s your Decision Support Database?
In our last edition, we explored the value of performance assessment, and how the timing of gathering data and the way in which the feedback is presented are key factors to success for your overall training implementation and for your training participants. In this edition, we take the next step into decision support databases, which use the accumulated performance assessment data to discern when individuals achieve competency or partial competency and overall training goals have been met, especially when sustainability is of significant interest.
Most importantly, decision support databases help you do just that – make decisions. Teamed with the coaching and performance assessment elements that we shared in our previous editions of our “Tools for Implementation” series, you can refer to a decision support database on a regular basis to check in with the long-term story of training implementation and to make course corrections along the way, if needed.
A simple version of a decision support database
The term “Decision Support Database” is quite a mouthful and can sound rather daunting to the average person. Let us de-mystify for you: You have a decision support database even when you track just a few simple performance measures in a simple Excel spreadsheet. For instance, if you are implementing Motivational Interviewing training, an extremely simple version of a decision support database might involve keeping an Excel sheet that tracks the numbers of open and closed questions that participants used…
Over time, you can watch this data in your simple decision support database (Excel sheet) to determine whether folks are making the shift from using closed to open questions. This useful “meta-information” then guides you in decisions, like what kinds of practice exercises to offer when the training group meets for a skill refresher – if the overall meta-data says that participants are primarily using open questions rather than closed, you could move on to another skill practice, like increasing reflections. But if your meta-data shows that closed questions are the primary kinds of questions used, then you can focus your skill practice to strengthen ease and familiarity with asking open questions instead of closed.
More complex decision support databases
Knowing how folks are doing with open and closed questions is good information, and if you are implementing for sustainability, you will likely find yourself wanting to track more than that! That’s when your decision support database becomes more complex and tracks multiple layers of information. For instance, instead of just tracking open and closed questions, you might also track all of the skills involved in reaching MI competency (e.g. open questions, closed questions, complex reflections, affirmations, no advice-giving, reflection-to-question ratio) so that you can see how your participants are faring in reaching an overall level of skill. In this case, you would likely employ an Excel sheet that generates graphs to visually show you all kinds of relationships between the different data elements that you have gathered.
The information gathered in a more complex database guides you to make larger decisions about what to implement next, based on how participants are doing and your overall goals for implementation. For example, in one state-wide project for scaling-up MI (50+ % of the officers achieving MITI-3 competency thresholds) a subset of officers expressed a great deal of frustration at one point. These officers had submitted many tapes for independent ratings and they were becoming discouraged that they still weren’t able to achieve competency. When coordinators looked at the data that emerged out of centrally collecting all the MITI-3 measures, along with other coding, training and coaching information for hundreds of officers a pattern was detected that explained a good part of what was going on. There was a distinct tendency with officers who had turned in the most tapes, to also be the officers with least phone or face to face coaching episodes. In another words, some officers in their eagerness to achieve competency, were submitting one tape after another without undergoing much coaching. Because the coordinators had a decision-support database they could refer to, once the above pattern was detected, the remedy was relatively simple… provide a better balance of coaching and coding.
The range and number of examples for the kinds of thing decision-support database can assist in is really extensive. Anywhere there are fidelity measures or performance assessments taking place, feedback and skill development are invariably involved as well. This makes for a very dynamic and seemingly chaotic system context – like a popcorn machine with an open top – unless a database is quickly established for tracking things. When the time and location of each corn kernel is identified before the heat is turned up, again when each kernel pops to highest point and then when it lands, this provides the basis for aggregate data, from which many patterns can be detected. When one sector (e.g., residential community corrections versus parole) requires a much longer cycle to achieve fidelity for a program, its time to look at the data and see what it says. Or if you have a group of people coding tapes and you want to see if they have similar inter-rater reliability rates by pulling out aggregate samples of each coder’s profile you will quickly see where the group’s strengths and weaknesses are.
Building decision-support databases can’t be done very effectively through large Management Information System (MIS) technologies. Getting a ‘job ticket’ and waiting through all the bureaucratic committee processes takes too long. Fortunately all the is necessary is an Excel spreadsheet and a little moxie to get started. Over time the data and the spreadsheet can be refined, defined better, and formulas can be utilized to automatically graph and report various indices out to the data. The key is to start throwing the spaghetti on the wall – gathering the data in ways that eventually are apt to be meaningful.
The difference when decision support databases are and aren’t used
Without a decision support database to guide your choices, you will still likely have outcomes from learning a new skill like MI, such as staff feeling better about some aspects of their jobs, or having less conflicts with their clients or having deeper insights about what is really going on with their clients. Your agency may experience some beneficial changes in the norms for interactions with clients. All of these are desirable and positive changes, and then using a decision support database can take your overall outcomes to the next level. Using the decision support database as your guide, you identify the level of fidelity to recognized measures and thresholds of performance so that you can reflect on the overall picture of outcomes for your MI learning process and make decisions about what you need to supplement and where you can celebrate where you are doing well.
At J-SAT we’ve benefited from using decision support databases to assess the outcomes of the coding and coaching process with our clients. If you are interested in Assessment or MI-Only coding and coaching services, please contact us at info@j-sat.com for more information.