Modern viruses (worms, trojans, etc.) can be almost unbelievably destructive. They can infect every pore of a system - dll's, registry, O.S. files. Everything.
And more importantly, modern viruses can can HUGE amounts of time to fix. And sometimes they can't be fixed. And that means they can be extremely unprofitable! When a new computer with a fast processor and all the software you need is less than $1,500, there's a limit to how many hours you want to spend "fixing" viruses.
A standard operating procedure is in order.
SOP Friday: Responding to Viruses
- Overview -
Unlike the SOPs we've discussed so far, this one is strictly defensive in nature. How do you restore the machine, keep the client happy, provide a timely response, and make money (or at least not lose money)?
More than anything, virus protection is most successful when you are very well prepared. That means the right hardware, the right software, the right configurations, the right customer training, and the right practices. All of that makes it possible for you to have the right response. Without adequate preparation, there may be no good response. Let divide this world of preparation so we can conquer it.
First, you need to lay the groundwork with hardware and software. If you're a managed service provider, your life gets pretty easy here. If you're not, then you just have to convince your clients.
Note: Some of these "policies" are really the essence of Standard Operating Procedures. We recommend one way of doing things. We push. We cajole. We quote the right tools, etc. We can't force a client to protect their systems. Which leads to one of my favorite sayings:
We can't care more about the client's network than they do.
Our managed services contract (see Service Agreements for SMB Consultants) specifically requires that the client have a good, working firewall that's under warranty or covered by a maintenance agreement. In other words, it's the latest and greatest, and can protect them from new attacks that show up unannounced.
It's amazing how effective hardware firewalls can be at detecting and stopping viruses - even the ones where clients are tricked into "installing" the Anti-Virus 2011 virus. Now, let's be honest. We're talking about a $750 firewall, NOT a $49 firewall. See the discussions around last week's SOP Friday discussion.
As for software . . .
This has two components. First, there's anti-virus software. This one is fairly obvious and takes very little convincing. The main decision is whether you're supplying and annual renewal or a monthly subscription. If you have annual renewals, you need ticklers to remind you to send the invoices.
The other piece of the software puzzle is Newer Programs. Old programs - like Microsoft Office 97 and Windows XP have some vulnerabilities that will never be fixed. Newer programs and operating systems are inherently more secure. Moving clients to the newer stuff is a never ending battle. We are constantly reminding clients that modern software is part of their security.
Hardware, operating systems, and software, must all be kept patched and updated. That means you need to have those processes as part of your maintenance plan, whether on managed services or not.
We do not currently image desktop machines. Our policy is that machines should be properly maintained, we limit our exposure to virus incidents, and we fix machines when a virus hits.
On related notes, it is our policy that we do NOT redirect My Documents to the server. We don't encourage clients to use My Docs. All information that's important needs to live on the server. Period. The server is on redundant drives and backed up. The desktops are essentially disposable.
Having said that, I have often heard people say that they image desktops and use them to restore from virus attacks. This sounds great as long as the desktop never changes. If you need to restore an image and then run all the updates since the image was refreshed, it may not save you any labor.
Imaging is a viable option. We just don't happen to practice it.
Note: We DO allocate space to let Windows store previous versions so we can roll back to before the Virus hit. That has saved our bacon more than one. Just remember to do it.
There are two kinds of client education related to viruses. First, there's education on your contract/agreement and what your response will be. Second, there's training on how to avoid viruses and what to do when one hits.
Our contact is very clear on this point: All maintenance, including all software installations, must be performed by an employee of America's Tech Support. So, when a client installs a virus on their computer, it is not covered by the managed services agreement.
Now, the truth is, we're going to believe the client that it's an accident and fix the first incident for free. But we're also going to make it very clear that they next one is on THEM. They'll get a bill for X hours at $00.00/hour. But we'll also make sure they know the next one will be for $150/hour.
Client education consists of emails, memos, newsletters, harping, haranguing, and whatever else we need to get across a few simple points:
1) You already have an anti-virus program. You don't ever need to install another one, no matter what pops up in front of you.
2) Whenever you receive an email with an attachment and you did not ask that person to send you that attachment, Delete It!
Period. I don't care if it's your mother or your boss. If need be, email them back and ask if they sent it.
3) Whenever you receive email with links that look urgent, do not click on them. Go to the appropriate web site yourself by typing the regular address into your browser (e.g., your bank). Log in. If there's an urgent matter that needs your attention, it should be flashing in front of your face. Delete the email.
4) If you're browsing the web and a window opens up by itself, click the Red X in the upper right hand corner. Do not click . . .
or anything else. Just click the Red X to close the window. If you feel violated, reboot your computer.
5) If you get an infection, log off of your computer. If you can't log off, restart the computer (force a power down and restart) and do NOT log on. We need the computer on to connect remotely.
The Bottom Line: Educating your client about your policies and their expected behavior will help limit your liability/exposure during a virus infestation.
Stand Firm by your processes and procedures. 99% of modern viruses are stopped by almost any anti-virus software . . . until the user clicks OK. In other words, it is almost always the user doing this to themselves. They need to understand that.
So . . . when you finally get a service ticket about a virus infection, what do you do? Here's a rough outline of our process.
1. As with any ticket, determine the urgency and assign a priority level.
2. Have a discussion with the client. Remind them about the policies. Verify the maximum number of hours we will put into fixing a machine before we move to billable labor. Request how many hours of billable labor are acceptable before the client wants us to stop working on the issue and simply re-install the O.S.
It is very important that you agree on limits to your time and to what happens when you reach those limits.
3. Connect to the machine remotely and log on in safe mode.
We do this with Zenith Infotech and LogMeIn. Other programs may provide similar functionality, including machines with Intel V-Pro network cards and LogMeIn.
If you don't have such remote access, then you'll need to be onsite. In either case, log on in safe mode. This will stop user-specific viruses from continuing to cripple the machine.
4. Attempt to clean the machine with your standard company-approved tools. These may include Trend, Symantec, AVG, Hit Man Pro, or whatever your decides is the best fit for you.
5a. If that appears to work, reboot the machine, log on as the user, and attempt to verify that the virus is gone.
5b. If that appears not to work, attempt to restore the machine to an earlier version running the tools built into the operating system. If you know the day the machine was infected, you should be able to restore to a previously working version.
6. If you believe the virus has been cleaned, apply all appropriate updates, and create a new restore point.
- Implementation Notes -
Implementing this policy can be very troublesome. Many clients insist that local users have admin rights. That's now always in their best interest. If you're losing money every time they get a virus, then it's not in your best interest either.
If a client allows themselves to be infected more than once, you really need to take them out of the local administrator's group. This might mean that the client needs to pay you to install a few programs here and there, but the cost is very small compared to a four-hour bill for fixing viruses.
As I mentioned earlier, an appropriate response means the right hardware, the right software, the right configurations, the right customer training, and the right practices. That means you need to really think through these processes and push them on to employees and clients every time there's a virus.
Note on "All You Can Eat"
I have never been a fan of "All you can eat" managed services. After sixteen years in this business, I know "all" some clients can eat is my entire company! Fighting viruses is a perfect example of that. You need to limit your losses with good processes and policies.
- Forms -
There are no specific forms for implementing this SOP. You might write up a brief description of the procedure and put it into your SOP or binder.
This kind of policy requires that everyone on the team
1) Be aware of the policy
2) Practice the policy
3) Correct one another's errors
4) Support one another with reminders
Your Comments Welcome.
Want to figure out how to make money with Cloud Computing?
Join the Cloud Services Roundtable today and listen a great series of podcasts!