Very recently I interviewed Jason Jones, CTO at OSSI-IT, an IT consulting firm. Jason provides IT services for Commonwealth Orthopaedics, an orthopaedic surgery group practice.
JK: Tell us a little bit about the size of the environment and applications you consider critical at Commonwealth Orthopaedics?
Jason: We have a total of 7 IT staff supporting about 600 people in 10 offices. Our environment consists of about 120 VMs all running on a Windows platform. In terms of the applications that Commonwealth Orthopedic uses are an Electronic Health Record (EHR), practice management systems, Exchange server, a number of SQL installations, accounting packages and systems, and handful of very critical applications that tie the systems together. For example, smaller, lesser known applications like Charge Capture that ties the EHR to the Practice Management system for the billing cycle. Although such applications may not mean much to a lot of people, we consider them as critical applications and rely on a handful of similar applications like that.
JK: Tell us how your journey began with Commonwealth Orthopedic?
Jason: We’ve been working with Commonwealth Orthopedic for at least 12 to 15 years. It started with maintaining their ambulatory surgery software. The software vendor partnered with OSSI-IT to help ensure that the ambulatory surgery software was running correctly.
JK: When did you help them figure out that Commonwealth Orthopaedics needed a monitoring solution?
Jason: I think Commonwealth has used SolarWinds for 8 or more years. It was originally brought in to really just look at up/down status for nodes. It was tied in with our Cisco routers so we can monitor our Multiprotocol Label Switching (MPLS) network. At that point everything was a physical server, so we didn’t have any virtualization when we were originally using it.
What brought them to monitoring was they needed something to monitor their new MPLS network after they switched from their point to point V1S system. They needed to understand what was going across the circuit, and what the response time was so we could manage their line of business application experience.
SolarWinds Server & Application Monitor (SAM) was brought into the mix shortly after it was made available, about 5 years ago. But we were using a hodge podge of monitoring solutions. We’ve had asset management, various states of Dell OpenManage and it was very clumsy. We had a handful of applications where we had no proactive visibility. We bought SAM to give a single pane of glass of application performance. Seemed like a no brainer shortly after we bought it. Without the single pane of glass, it took time to work between apps, and we had almost no proactive monitoring.
JK: After implementing SAM, did you see the service levels improve? How did you see the efficiencies of the IT shop improve?
Jason: The service levels improved overall. I think a couple of the biggest advantages to the service levels was we had less reactive instances, we could see when disk space was getting low, we could see when processes had high utilization, and could respond to issues faster. Also, when we deal with a number of hardware and software vendors, SAM has allowed us to cut through finger pointing and much more efficiently determine whose issue it was – whether it is a SQL server issue, or a query, or a hardware problem – we could tie everything together with SAM. We didn’t have to chase down everybody and were able to determine where the problem was.
It definitely also helped us to better plan for software and hardware upgrades, so we could get a better view of user experience. We could see where the deficiencies and performance issues were across our hardware and software assets and make educated decisions about where to spend our money in the future.
JK: Can you give a more specific example of the finger pointing aspect you’re talking about?
Jason: We had an instance a few years ago where we had done an upgrade with our practice management software and it was dreadfully slow. We were still new to using SAM and we had reached out to our practice management vendor and they were pointing to hardware, and then it was a Microsoft issue, then a SQL issue, and we literally went through about six months of going back and forth and trying different things. Only when we sat back and properly started monitoring our environment using SAM, we noticed all the different processes, and could nail down the issue which had to do with the way the SQL database was setup. In using SAM, we were able to absolve hardware of the issue, and narrow down the problem quickly.
Another instance that was more recent - we have a charge capture system that gets information fed from the practice management system. The charge capture system is used to generate information that goes back into the practice management system, and it also goes back into the EHR. So there is this three way (six way total), communication between all these systems. We use SAM to monitor all the integration pieces and all the application components. For example, for each of the 3 pieces of software - 2 have database components, 2 have SQL servers, 2 have IIS web based components. For the integration pieces, some of them are TCP ports, and shared bio folders where HL7 data passes back and forth. We use SAM to monitor the availability of the ports, application components (like SQL server, IIS), and whether or not information is getting backed up. We also look at SQL database tables using the AppInsight for SQL feature and custom user experience monitors in SAM.
Any time there is an issue with information missing from one application, we go into SAM. Just this morning, there was an issue, we looked at SAM and could instantly see that the integration systems were working well. We could then narrow down the problem to the charge capture application - it wasn’t sending out the information. We can notify the vendor and do this all before anyone notices there is problem with the system.
JK: Did you look at any other vendors before looking at SolarWinds?
Jason: We’ve been using SolarWinds for a while so it kind of made sense to continue adding components. We use Server & Application Monitor (SAM), Network Performance Monitor (NPM), Network Configuration Manager (NCM), NetFlow Traffic Analyzer (NTA), VoIP & Network Quality Manager (VNQM), and Virtualization Manager (VMAN). We’ve got a pretty thorough suite.
The only time we shopped around was when we were looking for a virtualization component. We looked at using Veeam once to monitor our virtual environment. We ended up settling on Virtualzation Manager because we knew at that time there would eventually be integration with the rest of the Orion portfolio so we could have a single pane of glass. Now that the integration is in place, the product is much better.
JK: Any other benefits or use cases you would like to discuss with regard to other SolarWinds products you use?
Jason: We really like (Network Configuration Manager) NCM because we have customized it to work with our Cisco routers, Dell switches, and varying hardware vendors. We really like using NCM because we’ve got it tied together and know that our configurations are safe, which can be a pain because there is no native or automated backup processes for any of those systems.
With Virtualization Manager, we like the consolidated alerts view. Going into vCenter, it’s obviously very robust for management, but it doesn’t do a good job of showing alarms in a manageable view. So we like Virtualization Manager for that.
We also like NetFlow because when we are troubleshooting an issue with intersite connectivity like we’re on a Metro Ethernet (Metro E) system, we can very easily go on and determine where different problems lie and we can almost instantly isolate who is chewing up what amount of bandwidth. So if we need to perform an upgrade, or if we are looking at upgrading circuits, or of we need to slap someone on the hand, we’ve got all that information available to us instantly.
For example, we had one site that was consistently slow – all apps were slow. From Network Performance Monitor we could see utilization was high. This always happened at certain times of the year. After we installed NetFlow, we found out it was one of the physicians watching the Tour de France. Because of that, we were able to put in QoS policies in place which has benefited us many times over.