Super User programs are a common piece of the implementation puzzle, but what is their purpose after go-live? Many organizations drop their Super User programs just after their EHR system is implemented. Leveraging your super users post-live can be a significant benefit to long-term behavioral change and system adoption. The key is tailoring the program to your organization’s specific needs and culture.
Sarah Rosebrock, Nordic Training Solutions Manager, sits down with Georgianna Kelly, Jerrie Rankin, and Kaitlin Kalina from Baylor Scott & White Health to discuss their system-wide program and how it supports post-live efforts. Tune in below.
Show Notes
[00:00] Intros
[04:56] What is a Super User and what is a Super User program?
[07:50] Are Super User programs vendor agnostic?
[10:07] The role that operations plays in the creation and maintenance of a Super User program
[12:42] Formation of the Super User team and the importance of its core statement
[17:48] ACES – the origin and meaning behind the name
[19:11] Importance of a Super User program
[21:33] Empowering proficiency among end users
[23:53] Why bring in Nordic?
[28:07] Guiding principles of the ACES program
[34:53] How to keep ACES engaged
[37:05] ACES have been the key to the challenges faced in 2020
[39:11] The secret to the ‘ooze’ – how to make your Super User program super
[44:02] Navigating the hurdles
[45:46] The most successful strategic move for the ACES program
[49:25] Accountability process to enable assessment of Super User engagement
[51:36] Lessons learned
[59:50] What’s next for the ACES program
Transcript
Sarah Rosebrock: Hello and welcome to this conversation about Super User programs. I’m Sarah Rosebrock, one of the managers with the Training Solutions team here at Nordic. Today I am joined by Kaitlin Kalina, Georgianna Kelly, and Jerrie Rankin, who, together, spearheaded the Super User program at Baylor Scott & White Health. Welcome everyone.
Sarah Rosebrock: Thank you for joining us, Georgianna. Could you introduce yourself and provide a little bit of your background?
Georgianna Kelly: Sure. Sarah, thank you so much for hosting this podcast. I've been working in the Epic space for more than 20 years in a combination of consulting and in FTE roles. I have extensive experience in assisting in leading healthcare systems in implementing and in optimizing Epic. For the past almost two years, I've been privileged to serve as the enterprise technical training and support director for Baylor Scott & White Health.
Sarah Rosebrock: Could you share with us a bit about Baylor Scott & White Health?
Georgianna Kelly: Sure. So Baylor Scott & White Health is the largest not-for-profit healthcare system in Texas, and one of the largest not-for-profit healthcare systems in the United States. We have more than 800 patient access points, and we've had more than 1.5 million patient encounters annually. And when you think about our geographic spread from north to south, east to west, if you were to lay the state of Indiana over that geographic spread, we would encompass most of the territory of the state of Indiana. So, in other words, we're very large.
Sarah Rosebrock: So, Jerrie, I would love to hear more about yourself and a little bit of your background so our listeners can learn about you before we chat.
Jerrie Rankin: Okay so, my name is Jerrie Rankin, and I am the Vice President of Operational Informatics for Baylor Scott & White Health. I report to the CMIO, Dr. Gina O'Brien. I have been in physician practice management for a long time, over 20 years, probably over 25 years, and was very deep into ambulatory operations with providers. About five or six years ago, I was asked to help be the operational lead for our Epic go-live, where we took 350 sites live in a big bang over 18 months.
Jerrie Rankin: So, at that time, I thought I was going to keep my operation duties and do the Epic work and as it got farther and farther down the road, that became less possible. So, I really embedded myself in all of the Epic work. So, after we got that done, and a couple of years into that, I was asked to join the Informatics team. And so it's been really, I think, a good thing. Operational Informatics is kind of a different title, but we were really trying to tie together all the operations with the functionality and really make that an effective role, so, that's a little bit of my background.
Sarah Rosebrock: Kaitlin, I'd love for you to introduce yourself and provide some of your background.
Kaitlin Kalina: Hi, thank you so much for having me. I started working with Epic in 2012 as the principal trainer, full-time employee at Scott & White in Temple and Round Rock. Then I spent five years consulting at various hospital systems installing Epic. Two of those years, I actually worked with Nordic. And a little over a year ago, I was hired as the ACES coordinator at Baylor Scott & White.
Sarah Rosebrock: Excellent. It's always nice to have a little green in your background. Can you tell us more about the role you play in the Super User or ACES program at Baylor Scott & White?
Kaitlin Kalina: As the ACES coordinator, I function as really a point person for the whole Super User program. I coordinate meetings with the Super Users, work with the PTs to consolidate training materials to build our ACES tool kits that we then give out to our ACES. I work with leaders to keep the ACES informed about the program as we evolve, and to make the program even better than it is.
Sarah Rosebrock: So Kaitlin, as we're talking about a Super User program, we may have folks out there listening who are unfamiliar with the concept. What is a Super User and what is a Super User program?
Kaitlin Kalina: Yeah. Super Users are people who are particularly adept at navigating the technology they use in their roles from day to day. They are people that their colleagues go to when they have a question, or they need help troubleshooting. To formalize a Super User program gives more structure to the support. Often organizations use Super User programs during implementation. Our ACES program is post-implementation, so our able, capable, effective, and skilled users support their colleagues during Epic upgrades.
Georgianna Kelly: Traditionally, Super User programs have been established as a part of an implementation. It's a very cost-effective way to support end users during the go-live and to ensure all of that subject matter expertise wasn't embedded in the consulting or the ATE team, they're supporting the go-live. So, most organizations would subsequently take upgrades every 18 months or so and tap into those Super Users, again, to support the end users. So, it would be every 18 months, every maybe 24-month cycle. Now, some Super User programs remain vibrant between those upgrades and some would really go into a state of hibernation. But today in the Epic space, having an established and formalized Super User program is increasingly important for a couple of reasons. One, Epic's posting changes quarterly. So gone are the days when you could wait 18 months or two years to adopt changes.
Georgianna Kelly: Two, many of those changes are not big-ticket items that grab an end user's attention like the switch to Storyboard. For something like Storyboard, we can say the way Epic looks will change so please pay attention to all of these various communications we're sending out. But for the less radical changes, it's really difficult to capture end user's attention in a newsletter or a broadcast message or posting on a SharePoint site, or what have you. So, a Super User can announce those changes at meetings, provide a demo of those changes if applicable, or whatever is the most appropriate means to communicate. And third, most organizations have moved past the implementation stage to the adoption stage where they're really working hard to get their end users to adopt the technology they worked so hard to implement, and this can take the form of going live with end user request enhancements, or implementing efficiency tools and functionality native to Epic.
Georgianna Kelly: So, having subject matter experts embedded with those end users to ensure those optimization efforts are adopted is really a key ingredient to the success of those initiatives.
Sarah Rosebrock: Actually, I think that it leads really well into a subsequent question that, I already know your answer, but as far as being vendor agnostic. So, I'm curious if you're okay speaking to a little bit about whether or not Super User programs are just appropriate for Epic.
Georgianna Kelly: I don't have deep experience anywhere but the Epic space, but I do have experience working in other industries and leading people through change, leading teams through change, I would argue it's always a best practice to empower your team to understand the change, make the change make sense for themselves, for their team members. If you empower your people and you trust your people, your people will absolutely, positively, amaze you, but it's really providing that structure, it's providing that formality, if you will, around here's the information, crystal clear communication. My team hears this from me all the time. Did you communicate? Did you communicate clearly? Was it crystal clear? So absolutely I would say, anytime you're leading a team through change, if you empower them with the information, they're going to do amazing things. They're most likely going to surprise the heck out of you and do the right thing.
Sarah Rosebrock: I love that as a leadership plan.
Georgianna Kelly: A Super User program, I don't know if it has different nomenclature based on different vendors. Probably not, but I would bet a large sum of money that every large healthcare organization is most likely pushing changes at a very rapid rate, not simply because their vendors are encouraging them or putting them on a particular refresh cycle, but just the state of the world today, just out of sheer necessity as healthcare organizations flex and meet the demands of COVID-19 and meet the demands of the consumerization of healthcare. I would say, it's absolutely crucial for healthcare organizations to be as nimble as possible and to be able to truly onboard and adopt change as quickly as possible.
Sarah Rosebrock: Thank you for that. You spoke a lot towards your operational background. I would love to hear how operations plays a role or plays a key role in the creation and maintenance of a Super User program?
Jerrie Rankin: Absolutely. So, I believe operations, it's vital. You really can't be successful with a program like this without it. I really don't see it as an IS-driven thing. I don't necessarily see it informatics-driven. Although I believe in informatics, we can provide the structure around what they need because sometimes they don't know. So, I think we can kind of help put that structure that accountability, the communication all around it for them. But then it's really kind of a team thing because we do need IS resources as well. You know, operations, I mean, it's very important for them that their staff stays up to date and with Epic quarterly upgrades coming, it's just really difficult to make sure that everyone is onboard, everyone is keeping up with what is happening. And so, by putting together a program like this and working with them, we're able to give them that structure so they have some confidence that their staff is aware of the changes, is staying up with the changes and it makes it worth their time to invest in it.
Sarah Rosebrock: Yeah. That makes a lot of sense. If you don't get operations to believe in it, then you don't get much participation, do you?
Jerrie Rankin: Well, no. And I think even if they believe in it, sometimes you still have to put together a structure that helps them monitor it and keep it in front of them because they've got so many other things on their plate, that it's the electronic health record. It's what you use, it should be fine. And sometimes they think once implementation is over, it's done. And so, part of our work is helping educate them that that's not the case. That is basically the beginning. And I think that that's just something that's... if they haven't been involved deeply with the Epic work, they just really don't have a concept about that and so they really don't understand the level of change, the pace of change, and their responsibility to make sure that their staff is up to speed because it really can't be information's responsibility.
Jerrie Rankin: It can't be IS's responsibility. People are going to respond to the people that they report to. And so, if I'm your boss and I tell you, "You need to do this," you're going to do it. If I get a random email from a corporate department that says, "Hey, we got this new fancy program. You should jump in." There's no accountability there, so the operations team really has to do that. And I believe as operators, part of our job is to make sure our people have the tools to do their job the best they can every day. And so, this is an opportunity to help them do that.
Sarah Rosebrock: Will you tell us a little bit more about the formation of your team and the importance its core statement plays while you're defining its scope of work?
Georgianna Kelly: Sure. So, our enterprise technical training team was formed two years ago by bringing together team members from IS, from our North Texas ambulatory area, and from informatics. We also were bringing teams together from Austin, Temple, and the DFW area, which was an adjustment for some of our team members as they learned how to collaborate with remote team members. Baylor Scott & White Health encourages each of us to develop a core statement that aligns with the organization's mission and overall vision. It really helps each team member to find his or her individual contribution to meeting our goals and in living up to our mission. So since my team was formed by bringing people together from different teams, I thought it was important for us to have a team core statement, something that announced to the organization what my team's purpose is, but also to provide the team with a common identity, one that wasn't grounded in their respective legacy department, something that was new.
Georgianna Kelly: Our core statement is we commit to empowering the adoption of EHR-related technology so that patients and members are treated by the most proficient users in the nation. I was purposeful about choosing three words in that core statement: empower, adoption and proficient. I was deliberate about choosing the word empower because training at Baylor Scott & White Health isn't required once the end user completes new years of training. So we can empower an end user to take advantage of all of their learning opportunities we provide, but we don't mandate that training. Adoption because unless an end user truly adopts the technology and understands the functionality and associated workflows and how to operationalize that technology specific to his or her role in the organization. End users will most likely find workarounds that may seem more efficient to them in the moment, but in reality, they probably are introducing inefficiencies and frustrations into their daily work.
Georgianna Kelly: And that's a slippery slope because we take changes to Epic every quarter. So, the further an end user moves away from adoption, the more challenges that end user faces as we introduce those quarterly changes into our production system. And proficient because the word has such a broad definition. Those of us who've been in the Epic space for a while are conditioned to think of someone achieving proficiency as a step away from Epic certification. It means you're okay in Epic. You have some skills, you have some street cred, but you're not quite yet certified. But outside that context, the word has a fairly broad meaning. Everything from, I'm able to do something to I'm the expert at doing something. So, I chose the word proficiency so that we were able to create a proficiency continuum to accommodate all skill levels, everyone from a new hire to an experienced analyst.
Georgianna Kelly: In each level of the continuum, we'll have learning opportunities associated with it so that end users can progress at their own pace. The first four phases of that continuum are able, capable, experienced, and skilled, hence the name ACES. When we have empowered end users so that they can adopt the technology and become proficient in its use, now the organization is really prime to maximize its investment in Epic.
Sarah Rosebrock: Just as a side note, I think you have a fourth key word there, which is nation. I love that in the end, you're saying the most proficient users in the nation, that's a lofty and amazing goal, right? Not every organization is going to say, we want our users to be the most proficient.
Georgianna Kelly: Yeah. We definitely don't shy away from big goals in informatics. Like I like to say, one of the reasons I absolutely love being on Jerrie's team is because every morning I wake up, her bar is set so high I can't even see it. It's just so far in the clouds or sometimes in outer space, you're like, how on earth are we going to achieve that? But you know what? Every single time the gauntlet has been laid down or the challenge has been set forth, the team has risen to the occasion. So yeah, absolutely. At the end of the day, we will have the most proficient Epic users in the nation. That's our goal.
Sarah Rosebrock: That's awesome.
Georgianna Kelly: Yeah.
Sarah Rosebrock: So, Jerrie, the Baylor program was named ACES. Whose idea was that? And what is an ACE or what does ACES mean?
Jerrie Rankin: Okay, so that's an interesting story. So, our technical training director, Georgianna Kelly had been working on a proficiency continuum. And this proficiency continuum has eight or nine steps in it and each one of them has a letter. And so, the first part of the proficiency continuum we had determined would really be what our Super Users would be. And so, the first parts of it are able, capable, effective, and skilled. And so, we were all around a table trying to figure out what we were going to name our Super User program, because at the time we had hospitals still going live, we were still rolling out. And so, there was a whole Super User program around the go-lives.
Jerrie Rankin: And so, our CMIO wanted to make sure that we distinguish this between that, we didn't want that to be the same, people to get those two things confused. And so, we're like, "Well, we should just come up with a name. This is a great time to come up with a name." And so, Dr. O'Brien looked at the continuum and she goes, "Well, the first four things here are ACES, let's just call it that." And so, that's how we ended up with ACES.
Sarah Rosebrock: It's nice when you have epiphanies right there in the middle of a meeting.
Jerrie Rankin: Yeah. Right. And somebody that hasn't been working on it had to see that because Georgianna and I had been looking at it forever and didn't even think about that.
Sarah Rosebrock: Fresh perspectives are always good. Why is a Super User program and why is ACES important to an organization like Baylor Scott & White?
Jerrie Rankin: Well, I think a Super User program like this is important for every organization, but specifically for Baylor Scott & White, our geographic spread is really unbelievable. We're from... Texas is huge and all the way from the north to the south. I think I read where our geographic spread is for Baylor Scott & White Health is the same as the state of Indiana. So, we have a lot, a lot of ground to cover. And with quarterly upgrades coming, we've bundled our enhancements in with our quarterly upgrades. So, it's a lot of change every quarter and with budget constraints the way they are, especially with COVID, we knew that there was no way we would physically be able to touch all of these locations. I mean, there's just no possibility with that. And people aren't adding to their training teams or really adding to any shared services and big health systems.
Jerrie Rankin: And so, we're like, "How do we do this smarter?" And so, in order to cover the geographic spread, I think it's necessary. I think one of the other really great things about it is that there are kind of unofficial Super Users in every location where you're at. There are people that... they get it and everybody kind of goes to them. And so, I think this offers a career path for them as well. And we know that if you're in a clinic, you know that Super User in that clinic, they know the doctors, they know the patients, they know the workflows of that clinic. They could look at the information and determine, yep, this applies to us, no, this doesn't.
Jerrie Rankin: So, they can kind of triage that, but yet they can still deliver a consistent message across the entire enterprise. And so, I think it's just kind of a slippery slope because without maintaining proficiency, our users are going to fall farther and farther behind and that just leads to burnout and turnover and frustration and things like that. And so, a program like this helps address that issue, but then it also helps give advanced knowledge to people and help us grow our employees in the use and the knowledge of the technology.
Sarah Rosebrock: So how does the ACES program support empowering proficiency among your end users?
Georgianna Kelly: So, our ACES are absolutely foundational to our strategy for our end users to adopt technology. And really, this is Epic or otherwise. So, we're starting with Epic in our ACES program, but we definitely have designs to leverage our ACES with other technologies. So, our ACES are a direct link to end users and hands down our most important communication channel to our more than 50,000 end users. So, with every quarterly release, our trainers create what we call an ACES toolkit, which essentially is a nicely packaged compilation of all of our training artifacts and communications specific to a department or an application. And the ACES are empowered and encouraged to take that toolkit, put an operational wrapper on it. That could mean identifying a change that will have a significant workflow impact and working through how to best operationalize it. We have 52 clinical informaticists and 46 trainers on the informatics team.
Georgianna Kelly: So, all of our deliverables are created to be at an enterprise level because again, that's 52 clinical informaticists, 46 trainers supporting over 50,000 end users. And because of that size, we're not able to develop communication specific to an individual clinic or department, but the ACES are able. It's a very cost-effective solution that ensures specific information is communicated in a clinical or operational context that then makes sense to the end users. It's not 50,000-foot level, it's 50 at the 50-foot level. And that's key because those communications ensure all of our end users maintain their level of proficiency and that they don't lose proficiency with a particular Epic release. We've created the framework and the structure to efficiently and effectively transfer knowledge to the ACES, and then we empower the ACES to take that knowledge, get put on the operational clinical wrapper and disseminate that knowledge to their respective team members. So, in essence, we're democratizing the communication process.
Sarah Rosebrock: Georgianna, you and I met when you asked Nordic to come in and do some of the prep work for the ACES program. Do you mind sharing with our listeners why you chose to bring in a third party?
Georgianna Kelly: Absolutely. So, in 2019, we're in the midst of bringing all of our North Texas hospitals live on Epic. At the conclusion of those go-lives, then all of Baylor Scott & White Health would be on Epic. It was in that context that we understood the need of an enterprise level Super User program. Now the organization had existing Super User programs. We had a Super User program specific to our North Texas implementation, we had Super User programs in some of our facilities, we had Super User programs in some of our departments, we had Super User programs in some of our clinics, but the organization lacked an enterprise-level unified Super User program. We also understood that we'd be taking Epic updates every quarter and that even if the changes were small, we'd need to ensure we communicated those changes very clearly and efficiently.
Georgianna Kelly: So, we didn't see ourselves being successful in those communication efforts if we maintained a decentralized multiple Super User program. So again, we have 52 clinical informaticists, 46 trainers, 52,000 end users. It would have been virtually impossible for us to communicate effectively, let alone efficiently, if we kept our decentralized program. So, it was really a combination of the workload associated with bringing our North Texas hospitals live plus the opportunity we had to take the best from the existing Super User programs and the desire to incorporate best practices from other healthcare organization’s Super User programs. And we didn't have an internal resource available to provide that kind of thoughtful and rigorous approach, so I turned to the consulting space for assistance.
Georgianna Kelly: And in my experience, some healthcare organizations stand up Super User programs almost as an afterthought. And I really wanted to devote the time and resources to put the program on solid footing from the beginning. In order to accomplish that, we needed a fresh set of eyes to take a look at our organization, our goals, our challenges, and provide us with options. I didn't want something cookie cutter to say, well, you're about the size of this hospital or this healthcare system so use this. I really wanted a company that I knew would devote the time and attention and had an internal structure where you, Sarah, could reach out to your peers, reach out to your knowledge library, and really pull those best practices. And it was important that we had someone provide that analysis who wouldn't process those options through a Baylor Scott & White Health filter.
Georgianna Kelly: In other words, I wasn't interested in, this is the way it's always been done, I was interested in, here's a best practice that could be implemented to your organization, again, based on your size, geographic spread, goals, and anticipated pace of change. I wanted something custom. And I was also purposeful about really having the framework fleshed out. A Super User starter set, for example, one that allowed us to get something up and running quickly but was flexible enough that we could make necessary course corrections without difficulty. And we've been successful in taking that framework you provided, Sarah, and building upon it. We've been able to really accelerate the development of our Super User program because of it. So, thank you, Sarah, for delivering exactly what I asked for.
Sarah Rosebrock: Yeah, you're welcome. That was one of my favorite projects. I enjoyed getting to know you guys and the discovery was great to see how the organization has come together over the years, coming from multiple organizations into one enterprise. So that was really exciting for me to see. Talk to me about the steps it took to get this far. What were the guiding principles that Baylor Scott & White Health adopted in standing up the ACES program?
Georgianna Kelly: Yeah. So, our guiding principles were grounded in stakeholder identification, incorporating the best of our legacy Super User programs and showing value from the beginning to ensure the program doesn't lose momentum. So, Sarah, you really helped us with the first one, stakeholder identification. So, we focused on implementing the ACES program in the ambulatory space first. Our ambulatory team members had been live on Epic for at least three years, making that group one of our most experienced Epic users in the organization. And that group had a legacy Super User program with many elements and practices we wanted to keep. So, we were able to start with it and we simply overlaid the ACES program onto something that already existed. So, in a way, that was the easiest part for us. Now the same could be said for access services, the contact center, and the revenue cycle team.
Georgianna Kelly: Those teams already had an existing Super User program, which we could almost just simply plug into. And then, Sarah, you essentially provided us with a roster of which areas had an existing Super User program and of those, which were following the best practices we wanted to incorporate into the ACES program. So again, we got ACES live in those areas first. Now for those areas without an existing Super User program or those areas with a Super User program that had atrophied a bit, stakeholder identification was of paramount importance to us. And with the organization of our size and geographic spread, we wanted to ensure that we didn't inadvertently leave any group out. So, in order to identify those stakeholders, Jerrie met with every operations VP to explain the intent of the ACES program, answer any questions they had and really to assuage any concerns that this would just be one more thing that they had to do or one more thing that was going to have a short shelf life, and really to get their buy-in by detailing the value prop to their end users.
Georgianna Kelly: And I've seen so many of these projects go off the rails because leaders didn't take the time and complete the step or didn't pay adequate attention to it. Rather than approaching this initiative as something coming from corporate, Jerrie was able to speak to them as someone who has been in their shoes, and she spent decades in clinic operations. But she also ensured they bought into the program by really explaining the benefits to them. Namely, having a direct tie into the communication channel so that their team members would be as prepared as possible for changes coming every quarter. That really got those VP of ops’ attention. As Jerrie said, many of them didn't even know that Epic changes were going to be pushed every quarter. So, they quickly understood the importance of, oh my gosh, I need to have some ACES because I want my team to be prepared and in the know.
Georgianna Kelly: Now those VPs of operations then ensured their directors, their managers would participate in the program. Now, something we did is we offered guidance around the characteristics of individuals who would be an excellent ACES and really gave them time to recruit or identify their ACES team. Again, throughout this whole process, we wanted to move with pace, with urgency, but we wanted to be thoughtful and we want all of our participants to be thoughtful. We just didn't want somebody that didn't have a whole lot to do in a clinic or a particular area say, okay, well, you're my ACES, or historically supervisors would be the ACES or the Super Users and they wouldn't have time to devote to Super User activities. So, identifying those characteristics was an important step for us, for sure.
Georgianna Kelly: And as we roll out the ACES program, again, we continued to be deliberate with our stakeholder identification and buy-in. So, we're live with ACES in the inpatient setting in our facility in Temple. Again, that hospital had been live on Epic for years, had an existing Super User program, and we were leveraging our clinical informatics team to help really drive that rollout in Temple and in other inpatient facilities. We'll be live throughout our organization by July 2021. We will have every clinic, every HOD, every facility live on ACES. Now, in order to show value from the start, we spent a significant amount of time determining our success metrics. Now, this is important to us because we're asking the organization to invest a significant amount of time in this program and it's important to show a return on investment.
Georgianna Kelly: So, to quantify the value this program provides to the organization, we decided to create a database to capture all of the activities in which an ACES could participate. Everything from weekly or monthly meetings to sharing the information found in the ACES toolkit with our team members. So, our initial success measure is based on participation. We're able to report back to operational leaders on which activities their ACES were completing and which they aren't. We're able to report to executive leadership which clinics, departments, facilities have an ACES and which don't. And thankfully, we have an experienced database administrator on our team so we're able to develop and maintain this tool ourselves. Once we have the data in the database, we intend to then expand our success measures, take a look at opportunities for improvement, take a look at opportunities where the ACES are hitting it out of the park and recalibrate.
Georgianna Kelly: So those are our guiding principles: stakeholder identification, identifying best practices in our organization, and showing value from the start.
[Commercial break]
Sarah Rosebrock: So how many ACES do you have in the program?
Kaitlin Kalina: Currently, we have almost 700 ACES separated out over about 400 departments, which are then separated out between miles and miles across the whole state of Texas.
Sarah Rosebrock: Excellent. There's no way that you could see them in person on a regular basis, how do you keep them truly engaged?
Kaitlin Kalina: You're right. Baylor Scott & White is a huge hospital enterprise. We have hospitals across the whole state, and it would be impossible to see everybody regularly or even to meet them at all. Two things that really keep us engaged are one, that we do have regular meetings. For our ambulatory group, we meet every single Wednesday. So, before each quarterly upgrade, we talk about the changes that will go live during that upgrade and give them our ACES toolkits. But on off weeks, when we're not talking about the upgrade, we talk about various Epic workflows that might be more complex. We give them information just to disseminate to their colleagues. Meeting regularly really does help keep them engaged. It builds a sense of community with one another. And secondly, we use Microsoft Teams. This has been a huge, great resource for us. We distribute information through Microsoft Teams and really, it's just an ongoing conversation. The ACES are able to ask questions of me, but also of each other.
Kaitlin Kalina: If there's someone who works in a family medicine clinic in College Station, they're able to talk to someone who is in North Texas, and they might have a different question or just how they're solving problems across the state. And it's been a really good way to keep them engaged when they would maybe even never meet each other otherwise.
Sarah Rosebrock: It's amazing how things like Teams help us in other programs and ways that we never would have expected.
Kaitlin Kalina: Yeah, that's been a huge learning point, during this time and during this year where we've been pivoting a lot of different situations that we definitely didn't expect. Having this technology already in place has been huge to be able to pivot smoothly, at least.
Sarah Rosebrock: So, this has been a year of pivoting. Do you feel like having the ACES program has helped the organization pivot to more virtual programs while we have been dealing with the differences of 2020?
Kaitlin Kalina: Oh, absolutely. We were just kind of at the beginning of ACES when this COVID shutdown began and so we already had ACES identified, we had our team structure in place. We had a few meetings, we hadn't had any quarterly upgrades yet, but just having the people in place and having the communication structure there, was a huge benefit when things were changing so rapidly. We were able to say, "Hey guys, I know this is a last-minute thing, but we have some new tip sheets on what our protocols are for screening patients, of where we're doing testing, of what PPE is and how to find it.” And we've been able to give that information to the ACEs who are then able to disseminate it to their colleagues who are working the front lines.
Sarah Rosebrock: How have they responded to receiving that kind of information through the program?
Kaitlin Kalina: I really think that the ACES themselves, and ACES are people who are able, capable, effective and skilled in their jobs and in using Epic, I think that having information that's relevant and timely and be able to have it at their fingertips, has been really huge. So obviously, we're a really big health system and we have information everywhere as I'm sure most people do. There's information on team sites and there's information on tip sheets somewhere and all over the place, and sometimes it's hard to find what's the most accurate, what's the most up to date. And through our team's channels of communication, we're able to say, we just made this tip sheet, it was just finalized and able to push it out to people right that minute when they need it most.
Sarah Rosebrock: Excellent. So, you mentioned a secret to the ooze. When you say that, it makes me think of Ghostbusters for some reason.
Kaitlin Kalina: I think it's the Ninja Turtles, just so you know.
Sarah Rosebrock: Ninja Turtles. Okay. Sorry. Yeah. I'm getting my background mixed up. But I'm curious, how do you make the program super and not just the same thing that the end users receive?
Kaitlin Kalina: Absolutely. Yeah, so everyone is going to too many meetings, everyone's going to meetings all the time, and we don't want to just be another block on your calendar that you can't do something else. Right? I really want our ACES meetings to be meaningful and to be effective. One of the huge benefits that we had was, Jerrie Rankin, our VP, was able to go visit all of the regional vice presidents to get their buy-in for the program. So, when we began, leadership already understood the importance of ACES and how it was going to help their end users, their clinics, and their departments be even better users of Epic and take even better care of their patients. Since we had that buy-in from leadership, the end users and the Super Users were even more excited to be part of the program. It didn't take a lot of convincing at all to get them to be excited about it, which was huge. And secondly, a thing that sets ACES apart rather than all the other generic meetings that we have on our schedule, is that I really try to give information as it's pertinent.
Kaitlin Kalina: And not something to say, "Hey, this is information that's going to happen in 10 years,” or just reinforce things that you already know. I want to give information that's pertinent and that's helpful today, that's going to help you do your job better today. In the situations of our upgrades, obviously we do this a few weeks ahead of upgrades and we give them something to take home, right? We give them a prize essentially for coming to our ACES program, we call it an ACES toolkit. And in these ACES toolkits, the PTs work really hard of putting together these toolkits that have maybe PowerPoints or tip sheets and just all kinds of great information of what's going to go live with the upgrade bundle. The ACES then come to our meeting and then take the toolkit to disseminate to their colleagues. They perhaps hang things up in huddle boards, or they give a demo during a lunch and learn. And that information then makes obviously the go-live go better, but it helps the ACES then become leaders, become the go-to person in their own clinics.
Kaitlin Kalina: So, it's not just another meeting. It's really helping them build their skills of being a trusted resource in their clinic, but also learning how to communicate more effectively perhaps. It's just really good skills building as well.
Sarah Rosebrock: One of the things that is always brought up with Super User programs is how are you rewarding them? How are you paying them? Do you feel like the ACES feel rewarded by their skills that they're growing or by the confidence level that they're improving? Is there something intangible that they're also feeling rewarded by?
Kaitlin Kalina: I sure hope so. I think it feels good to be at least acknowledged for your participation, your knowledge that you're a leader. You're one of the people in our clinic that we can go to and we can ask questions and we know that you're going to know the answer or at least know where to find the answer if you don't know it. We wish we could actually pay them in money. That's not really part of the budget right now, but we are looking into ways that maybe we could do career laddering or some kind of badging that we could in less formal ways just to give them some kind of acknowledgement. We do have lanyards for ACES that people can pick them out of a crowd and say who they are and get some kind of pride from that as well.
Sarah Rosebrock: It's nice to have a sense of pride in something that's an additional duty for you on a daily basis. That's really nice. So, talk to me about the steps that it took to get this far. It sounded like there were stops and starts and a few false starts. How did you navigate those hurdles and how did that go?
Kaitlin Kalina: Yeah, so when we started, we had all of that pre-work that you did, Sarah, thank you so much for it, of finding out what the need was for the Super User program in our hospital system. And the places that already had existing Super Users and kind of figuring all that out. We had just some strategic ideas of how to implement. We did decide to first do outpatient only, which was good because they already had some kind of Super User program existing. Our first bundle that was going to go live, that we were going to have our ACES support with the new model was in April. So that happened right while everything was shifting because of COVID. So, we didn't have the exact kickoff that I would have loved to have. So, we ended up having, we just pushed out our ACES toolkit and did a lot of ad hoc question answering. And were able to pivot because we did have that structure already in place and were able to give them their toolkits and able to support them through that upgrade. And we were able to support them and they were able to support their colleagues as there was already enough stress going on during that time.
Sarah Rosebrock: I've heard you mention in the past that it's taken some strategery, so to speak, to get the program going, what has been the most successful strategic move for the ACES program?
Kaitlin Kalina: I mentioned it before, but I think one of the biggest strategic moves was Jerrie Rankin visiting the regional VPs and the leaders to really get that buy-in from the top down. Not that people were forced to be part of the program, but that leaders were able to see the value in it and that the end users and the Super Users didn't have to convince their bosses, convince other people that this was important. That if a leadership saw it from the top down, that it was easier to get buy-in from everyone. So, it didn't take as much convincing to get people to be part of the program. And then the ACES themselves had an easier time to carve out space in their schedule to attend our meetings and to share the information with their colleagues. Because we had that buy-in from leadership down, it made things a lot more smooth. And secondly, another great strategic move was to do outpatient first.
Kaitlin Kalina: So, we have obviously a huge geographic spread, our outpatient areas have been on Epic for several years now. So, it was much more stable, and they were looking for someone to help them just optimize and get better, and the end-users, the ACES themselves, were really comfortable with Epic. While our inpatient departments, our hospitals, we just, in June were just fully live on all of our hospitals, inpatient. So not everyone has been on Epic long enough to really feel comfortable that people would say, "Hey, I'm ready to be a Super User," because they haven't necessarily been using Epic that much or that long.
Sarah Rosebrock: That makes a lot of sense. You mentioned the buy-in from a leadership perspective to start with. I think that one of the hurdles that a lot of organizations deal with is Super Users actually having the time to participate. And it sounds like you guys have found a way to help ensure that Super Users have the time to participate and are involved in a way that it makes it impactful for everyone.
Kaitlin Kalina: Yeah. And we send out surveys after every bundle to try to see like, "Hey, do you feel like you've had enough time?" If no, then maybe we can talk to the leadership to say, "Hey, can we give a little bit more time to our ACES to help support this?" But I truly feel like because it's been a top-down, just that everyone's on the same page of how important ACES is and how important it is to get this information out, that leaders are more likely to give time and to support the program, that they'll see that if the ACES are able to do their role of supporting their colleagues, of giving the bundle information, if they're able to do that, then go-live will be smoother.
Sarah Rosebrock: What accountability process has Baylor Scott & White implemented to enable operations to assess their Super User engagement?
Jerrie Rankin: So, I'm really glad you asked that question, Sarah, because to me coming from operations, it was very important for us to put together something for them to hold accountability to their staff. Because again, like I mentioned, really operations has to do that piece. And so, the goal was to make it as easy as possible for them so that they had one place to go and they could go check and see, do I have Super Users where I need them? So, we created the dashboard and then this dashboard will basically allow them to see all of their locations, they can sort by locations so they can see all of their locations that they're responsible for. Do I have a Super User and is that Super User participating? And for part of the participation, some of that is self-reported, but are they attending the meetings? Are they sharing the information? Are they doing huddles? Those types of things that they can report into the dashboard. And so, it's very easy for the operational leaders to look at that and see if my bases are all covered. And my thought around that too is that a lot of times we get escalations from a clinic, we didn't know this, we didn't know that. And a lot of things that comes back to as well, what about your Super User? And they'll go, "Well, I don't know who my Super User is." So, part of that is going to be surveys as well, where we survey the end users and say, "Do you know who your Super User is? Are they effective?" And then also surveying the Super Users, "Are you getting the information you need? Is operations giving you the time that you need to do this role?" So, we're going to put all of that stuff in the dashboard so that the leaders have basically all the information they need to make sure they're managing this program effectively.
Sarah Rosebrock: Has that required any special tools to create those reports or that dashboard?
Jerrie Rankin: Yes. And bless Kaitlin and our technical people, because I keep saying, "Oh, but I want it to look this way, and I want it to look that way, and we need to add this." And so yes, they have worked on that.
Sarah Rosebrock: So, you’ve come this far, what are your lessons learned about the Super User program?
Jerrie Rankin: Well, I think we still have a lot more to learn, but for now I think one of the big ahas for me was when I went out and met with all the operational VPs to tell them about the program, sell them on it basically. And I would say that it was effective because of my background, because I could speak their language. What I found surprising was that they weren't really even aware of the need for this continuous learning. I think very few of them, maybe one, even knew that we were going to be having quarterly upgrades. So, that shocked me. I mean, because we eat, live, and breathe it. So, for us it's like, "Well, how could you not know that?" But they're dealing with a gazillion other things, the finances, and the running of the clinic, and now COVID, and all these other things. And so, I don't think they even understood what was all coming and the amount of change and how that was going to be our new normal. And so, that was really enlightening. And so, that kind of helped me sell the program because they were like, "Well, we don't know what to do. We don't know how to keep people up to speed." And so, by me coming in and saying, "We know that. We gotcha. We're going to have the process. This is the process. You're going to get this information. It's very regimented." I think a structured approach is what we learned is very, very important so that users know exactly what to expect. Managers know exactly what to expect. They know that if a change has come in and something's a problem, they know they go to their Super User, that Super User should have that information.
Jerrie Rankin: And so, I think you just can't turn it over. You can't say, "Here's a Super User program and here guys go take it and run with it." We learned that we had to have a full-time coordinator because they're just not going to own it with all of the other opportunities that they have and all of the things that they have that they're working through.
Jerrie Rankin: So, I think some of the other things that we've learned is just the continued challenge around the investment and the time. Everybody, especially with the COVID and the pandemic stuff happening, there's budget cuts all across systems. They're doing more with less. Now they're doing a lot with things with virtual visits and things like that. So, this really is an investment in your people and working with those teams to make them understand that this investment is really worth it.
Jerrie Rankin: And I think as we succeed in the program, then we'll be able to show that. So, there's that and then there's also, I think just helping them learn that the more proficient their team is, the more positive the downstream effects are going to be because they're going to be doing things right. They're going to be capturing charges. They're going to be doing all of these things in a timely, efficient manner, which is going to make everybody happy in the clinic, and it's also going to make it run better.
Kaitlin Kalina: The biggest thing I've learned is that it is not possible to communicate too much. Letting people know what's coming, giving them the toolkits to share, and just asking and answering questions has been a huge, huge win. Just over-communicating I think is one of the biggest lessons. And additionally, having a good database with who's in it in the departments and who's participating is absolutely valuable.
Georgianna Kelly: With our transition to a quarterly release, a strong Super User program is vital to our organization. Again, I will remember the days when organizations would take an upgrade and sometimes two, and at one organization, three, every 18 to 24 months. And those organizations, in essence, you were implementing Epic again. In between those upgrades, Super User programs would typically go into this state of hibernation and then be resurrected for the upgrade go-live. Well, by that point, you've had team members leave, team members added to the team. So, it was almost like a fire drill even identifying who your Super Users were. Simply not possible today. Even though the value prop in Epic pushing updates every quarter is the scope of change will be smaller, meaning the impact those changes should have on end users shouldn't be high, the sheer number of changes included in each release can be substantial.
Georgianna Kelly: So, in our last release, we had more than 120 release notes for EpicCare Ambulatory that had some level of end-user impact. Now, none of them were particularly high other than Storyboard, but 120 changes that were going to impact just EpicCare Ambulatory, that's a substantial number of changes. So how we would communicate those 120 changes without an effective Super User program, I'm not sure what that solution would be. So again, for us, we want to make sure our end users can at least maintain their level of Epic proficiency and then truly adopt any of the new features and functionalities that could then increase their proficiency in our main tool, for that is our ACES program. I've also learned that the source of communication matters. The open rate for our Epic newsletter is about 1%. So that's pretty easy math to do: 52,000 end users, only 1% is opening our Epic newsletters. Not reading, just opening. So, we obviously can't just include all of our updates in a newsletter and call it a day. And the feedback I'm receiving is when communication comes from a source our end users know and not from an anonymous corporate group, we have a much higher success rate in getting our end users to hear and see that communication. If somebody were to tap you on the shoulder and say, "Hey Sarah, I really need you to pay attention to these five changes," and it's somebody you know and trust, you're most likely going to pay attention to that person, versus having an Epic newsletter drop into your email, and if your email is like mine, you can get a tsunami of email every day and you could easily miss it.
Georgianna Kelly: And adoption happens at the grass roots. I mean, I can certainly empower and enable it, but in ACES, working in that clinic or department, someone who can translate the technology for that specific end user, that for us is the key to adoption. Another lesson learned is effective and efficient knowledge share is key. Super Users are able to tap into existing huddles or meetings rather than asking operations for additional time away from bedside or clinical care. And with the exception of something like Storyboard, again, the changes to Epic just aren't necessarily attention grabbing. So how do we capture our end user's attention with that many small niggling changes coming their way? Again, a vibrant Super User program helps us meet that challenge. And hands down, operations has to be invested. A Super User program is at its core an investment of time, and in any business, time is money.
Georgianna Kelly: In an organization with hundreds of Super Users, that is a significant investment. One of the absolute smartest things we did at the outset of our program, again, was having Jerrie meet with the VP of operations, again, to explain the program in detail, the metrics that we'll use, assurances that we're going to be there, we're building something that will last, we're not going to go into hibernation. And I would certainly identify that one activity as a best practice for any organization to starting their Super User program journey or contemplating a reboot to consider.
Sarah Rosebrock: It sounds like you've learned a lot so far. You mentioned a few ways that the program is going to expand. Could you speak more about what is next for the program?
Kaitlin Kalina: Inpatient’s next. So, we have gone live with one hospital using ACES, in Temple was our pilot and it went well. So, over the next several quarterly upgrades, we're going to be taking a few hospitals live with our ACES program until, I think September next year, we'll be fully implemented in ACES and then see what's next from there. It is very exciting because there's just so many places that this can go. Right now, we're focused on bundled upgrades, but as you guys know, Epic is always changing and obviously healthcare is always changing. And having people with their boots on the ground, in the clinics, who are using the technology and are seeing the patients, the people who are there, it's so great to have them there to be our eyes and ears, but also to disseminate information too. So, I can see it going in a thousand different ways and I'm really excited to see where it goes, to be along for the ride.
Georgianna Kelly: In the next 24 months, I have a few goals for the program. One is to embed the ACES program into selected career ladders. Technical fluency is an increasingly important skill set for clinicians and operators in healthcare, and rewarding the achievement of a certain level of technical proficiency will help to highlight that increasingly important skill set. Now, it doesn't necessarily mean additional pay. It could mean that, and it would be nice if it means that, but in addition to pay, we can recognize that recognitions at ACES could come with a different job title. It could come from recognition by senior leaders for excelling in the ACES role, pins, T-shirts are huge in our facilities. We have days where we can wear T-shirts. So, T-shirts really resonate with our end users. Or a feature story on our organization's main internal website page.
Georgianna Kelly: So obviously, money's nice, a raise is nice, but we're giving thought to other ways where we can recognize our ACES. Another goal is to hold a virtual ACES user group meeting. Again, really just to encourage relationship building and enable the cross-pollination of ideas and best practices. Third, want to involve the ACES in onboarding activities to ensure our new team members get on the path to adoption early in their tenure with our organization. I was actually meeting with a colleague who works on the operations improvement team and she was talking about the exit interviews. She was reviewing HR data from exit interviews, and it almost broke my heart because many of these team members who left the organization in 90 days or 120 days, they were leaving the organization mainly because they couldn't reconcile the technical skills needed with then the operational skills.
Georgianna Kelly: So, in essence, they couldn't operationalize the technology, they couldn't keep pace with all of the changes coming. Not necessarily the technology changes, but some of the workflow and process changes that are then driven through the technology. And so, I thought, holy smokes, we're not doing a good enough job in preparing these team members really to weather that first 90, 120 days as they're building that muscle memory, and they're either getting used to these new processes and workflows. So, hands down, we need to do a better job on onboarding our team members and I see ACES as playing a significant role in that endeavor. And then I hope to get some selected ACES involved in our testing cycle so that we can really capture the voice of the customer before going live with an upgrade.
Georgianna Kelly: Our informatics team does a great job of identifying those Nova notes or enhancements that will have high end-user impact in terms of changes to workflow, but who better to truly assess that impact than our end users who are going to have to be using it day in, day out. And again, that would really offer another rung, if you will, and a career ladder for those who may have hopes of joining IS or joining informatics eventually. And then I hope to use the data from our ACES database and again, compare it to our HR team member surveys and determine, is there a discernible link between the two? So, do we see a correlation between those clinics and departments with strong and active ACES and those areas’ HR satisfaction surveys? I would wager a large sum of money that that's exactly what I'm going to see.
Georgianna Kelly: And then finally, to expand the scope of the ACES program so that ACES are leveraged to communicate and support other changes. So, Epic's just one application of more than 300 in our application portfolio. So how can we package all of the changes hitting our end users so that the communication for those changes are meaningful, connected, and easily digestible, so that, again, they're adopted and not simply tolerated?
Jerrie Rankin: So, we have lots of ideas. I think we want to get more specific in our dashboard and our accountability reporting. We want to do a quarterly follow-up with leaders. So, as we get that information a little bit more solidified where we have a good comfort level with it, and we get through a couple of upgrades, I would like to do some quarterly follow-up with the operational leaders to show them stats, how their clinics are doing like, "A hundred percent of your clinics have a Super User or 80% do and they're in these buckets where we don't have them." Things like that.
Jerrie Rankin: Provide them with survey information. Would love to come up with some data that helps them see the benefits of the program. So, we'll be working on that. I think one of the other things that we're going to be working on is a reward structure. So, one thing I heard loud and clear when I was talking to the operational vice presidents was that they were interested in rewarding their people for being an ACE. And so, trying to figure out what that looks like, whether that is financially, whether we can do something there, whether we create full-time positions or one thing we've talked about is even creating a badge process in our LMS.
Jerrie Rankin: So, as they go up the proficiency continuum, they get able, they get capable. They get skilled. They get experienced. They earn badges all along the way. And that's something that we can do without any cost. And I think it was important to the operational leaders because this Super User process has been started and stopped so many times. And I'm sure that's very normal with a lot of organizations. They start, they stop. It fizzles out, whatever. And so, they really wanted to make sure that we really built a structure that would survive and that would make people interested in being part of it. So, that whole reward structure is something that we're going to look at.
Jerrie Rankin: And another thing I think that I would love to see, is some sort of a user group meeting, because we have over 500 now and I don't know, Kaitlin probably has more updated numbers than that, but that was just in our ambulatory world. And now we're moving into inpatient. And so, we're a big organization, I think a user group would be super fun. I don't know that we'll ever be able to get everybody together, just given our spread, but maybe do something virtually. We’re all getting really good at that right now. And so, maybe doing something for those users, kind of to reward them for that work and making them feel special for the role that they have, because they really are. I mean, they're kind of our extension of what we do.
Jerrie Rankin: Those are some things and obviously we'll constantly look for ways to improve. And that's another reason that I want to get with the leaders, is to hear from them, to see if there's anything else that we can do to make it better.
Sarah Rosebrock: Those are some great thoughts. Kaitlin, Georgianna, and Jerrie, thank you so much for joining me today and for sharing your thoughts and expertise on setting up a Super User program.
Georgianna Kelly: Thank you so much for having us.