Poor infrastructure design is a problem.  One that is further complicated by the very consultants trusted to avoid this very problem.  As we are currently seeing in a recent cyberattack in Atlanta, where they have to set aside more then $2.6 million for the recovery of a ransomware attack.  This attack took down a sizable portion of their infrastructure, and could have been avoided.

 

Many IT consultants use the same systems for every clients.  Most common are the all Microsoft shop, those consultants that have a Microsoft solution for every problem.  One of the key benefits to this setup is that there only needs to be a single skill set to maintain these systems, and that is a benefit.  However, we have known for a long while now that to avoid crippling attacks it's best to have multiple systems in place.  That way if a virus or attack takes down a specific system, your organization has other systems in place that will be unaffected.

The Atlanta situation comes from two main issues, failure to keep systems updated and depending largely on a single platform setup.

Microsoft products, while being very powerful and feature rich, have the problem of having many security issues.  To their credit, Microsoft has spent a lot of money over the last 15 years to create one of the best security audit and response teams in the world.  When a zero-day is discovered they can quickly identify and patch the issue, rolling out and update with remarkable speed.  The patch still needs to be installed.  This is by no means a simple feat.  Depending on the size and complexity of the network the rollout could need to be tested on servers that replicate the internal network.  This is done to make sure that the update does not interfier with other software or services.  Once the patch has been deemed safe it needs to be carfully and methodically installed throughout the entire network.  Patches sometime cause problems, which need to be either rolled back.  Or, in extreme cases, causing the whole server to be wiped, then receive a fresh install of the system; restoring data from backups.  It's because of all this extra work to keep the systems updated that consultants love Microsoft.  The product creates many billable hours of work in maintenance.  And when the system is maintained in house, there can and probably will be problems which will result in many hours of billable time for consultants to fix the issues.

The other issue Atlanta faced is that they relied mainly on a Microsoft solution.  If their network was dividiid between Microsoft, Linux, Apple, Oracle, and other solutions this problem would have been much smaller and less costly to fix.  The downside to mixed systems is the cost of maintenance.  This requires multiple specialties to be available, either as in house employees, or from a consulting firm such as the Yaniz Corporation, or multiple consulting companies.  However the total cost of ownership (TCO) would still be much less then $2.6 million they are having to pay.

There is a piece of advice I received when I started my first IT company in 1999.  "It costs nothing extra to do the project correctly the first time".  Since then I have seen many clients that have made decisions that were not inline with their business, causing them to either scrap the project to start over or modify the project in process which is very costly.  Creating a budget for an IT project can be difficult.  This is why it's important to hire a third party to work with your company to establish needs.

Update June 6, 2018

It looks like Atlanta will need an additional $9.5 million dollars to recover from the "SamSam" ransomware attack they faced in April, and the costs probably will keep rising.