How many of you that have brought in external consultants for some type of security engagement felt like you paid a lot of money for something you really didn't understand? Or better yet, how many of you have brought them in and felt like after they left you had less of an understanding of your environment and what your true risks were? It seems as though its becoming standard practice for a lot of groups to test for a few days (or simply run automated tools), crank out a templated report, and give a short presentation at the end of an engagement without detailed guidance for making the world a better place. Is there any value in this? Maybe, but for what you've likely paid not NEARLY enough.
While many would argue that our job security lies in customers knowing as little as possible about what goes on, I argue that you can achieve the same level of job safety as well as build additional trust and loyalty by not leaving them in the dark. Here is what I consider "leaving your customers in the dark":
1- Using misleading contract language to give the illusion that you are performing something a lot cooler than you really are. Not to mention billing for 30 hours of "reporting" when you are really having your consultants use an automated report generating tool.
2- Stating that your "methodology" requires performing several thousand "unique" checks" (doesn't Nessus/WebInspect/Appscan do that?? hmm). When asked about your methodology, you state that its "proprietary, super secret stuff".
3- Delivering a cookie-cutter report pulled directly from the output of these same tools, without ANY level of analysis or correlation of threats. Correlation example- Threat A is kinda sorta bad, and so is Threat B....when they both exist together, this is REALLY REALLY BAD. That additional layer of analysis and articulation to a client could mean the difference between them getting owned or properly prioritizing their mitigation tasks.
4- Not offering meaningful guidance for risk mitigation strategies, or better yet, how to prevent them from happening over and over.
Getting back to the education part, based on the 4 steps above, how much do you think this client learned throughout the engagement? Answer: not much. Not only do they have absolutely no idea what you did for 40/80/120 hours, they don't know what they are doing right. All they were likely told is "you have 38 instances of XSS, validate input and HTML encode output." No custom guidance stating how THEY should validate their input (what if they have to allow HTML tags...guess stripping all special characters goes out the door). Also, don't assume that just because there were no findings in a specific area that they'll figure out on their own that they did something right. If they have a particular area or approach to securing something that works extremely well, explain it to them so they continue to do it this way!!
Also, unless you are rolling your own in-house stuff that you don't want to release to the public (which you DO have the right to keep "proprietary"), give them advice on resources they can leverage in order to continue to build upon the security improvements that they **hopefully** gained through the services you provided. If you tell them that your toolset is "proprietary" even though it consists of Burp, Nikto, and ::insert_scanner_here::, you are truly doing them a disservice.
I guarantee that if your customers trust you, see the value in what you provide to them, and most importantly they IMPROVE year after year (read that again: IMPROVE), they'll keep coming back. Stop believing all of the "cost per vulnerability found" stuff. If you are using the number of vulnerabilities your consultants discover as your sole criteria for bringing them back, you are making a big mistake. Instead of saying "wow, finding 20 instances of XSS was only $10k", say "wow, we had 7 less XSS findings than last year, and now we truly understand the threat vectors of these vulnerabilities".
2 blog entries tonight...doesn't happen often =)
-Jack
2 comments:
"Not only do they have absolutely no idea what you did for 40/80/120 hours, they don't know what they are doing right"
I agree with everything but this. All of my engagements are met with limited interest. My clients aren't interested in understanding what it is we do. We make ourselves available for any guidance they may need, but they rarely take us up on our offers.
In relation to what they've done right? They don't pay us to give them a participation trophy and a hug. They're paying us to find out what they're doing wrong. The limited knowledge we have about the company prior to engagement prevents us from knowing a lot about their business needs, and the full scope of what they're trying to do as a company. How can we possibly know that they have one unruly client that requires inputting HTML into some online form? All of this must come out in the consultation after the engagement.
And I'll disagree with what you disagreed with =)
To be fair, not all clients are genuinely concerned with learning a whole lot. Some are, some aren't. Some want to improve their security posture, some want a check mark for compliance reasons.
HOWEVER, regardless of whether they care or not the approach should be exactly the same. If they don't take you up on your offer for an onsite or WebEx presentation demonstrating how you were able to compromise their systems or detect a particular issue, the deliverables should still contain as much detailed information as possible. This can only come from learning as much as possible about them, their environment, and ultimately their business needs throughout the engagement.
In regards to letting them know "what they did right", I think it has less to do with giving them a pat on the back or a "good job, tiger" and more with helping them to understand their own environment a little better. Some organizations may legitimately understand the security benefits of using prepared statements vice not using them to build their SQL queries. Some may have simply did a cut/paste of a code example from a random website. So lets say you determine that although they aren't filtering or sanitizing ANYTHING, your SQL Injection attempts all fail because you can't break out of a prepared statement. In this case, you could argue that they simply got really really lucky. If the code examples their entry-level developer used didn't do it the right way, they would have been riddled with tons and tons of dangerous holes. By neglecting to help them identify what they did right in this instance (which to some other groups may very well be painfully obvious), you are helping them on their way to developing repeatable GOOD development practices.
In regards to knowing if they may or may not require the need to allow users to create their own rich content via HTML and Javascript, this is something that should be identified in the pre-engagement phase. Unless you are tasked with performing an absolutely 100% blind test, I don't see any reason for something like this to not be identified prior to an engagement. Yet, it still happens. How could anyone say that its truly possible to perform a guided, thorough assessment of an application or networking infrastructure without first understanding the business purpose it serves, as well as some information about how their users actually make use of the application? Yes, you can certainly test for the common vulnerabilities without knowing that information. However, to truly put yourself in the mindset of someone out to cause damage to their organization, knowing the additional tidbits can mean all the difference in the world.
To sum it up, are we really concerned with finding a bunch of vulnerabilities that they may or may not fully grasp the impact of, or are we concerned with helping our clients achieve long-term improvements?
Post a Comment