Discuss this article in the ZeroFlaws forums

Over recent years it seems that employers and organisations are increasingly looking towards standardised testing to dig into people's capabilities and personalities. Companies often ask prospective interviewees to take a verbal reasoning test before even getting an interview, and at the other end of the scale MENSA is beloved of the IQ test to control membership (although one might wonder why people with high IQs need to pay a £45 membership fee to access a social network).

What does all this have to do with security? That's a very good question. Before I answer it, take a look at the question below. If you've ever had the pleasure (or perhaps misfortune) of taking an IQ or verbal reasoning test, you might have come across it in one guise or another. It's a classic "can you interpret the question's meaning correctly?" poser. Here it is in true IQ test format:

"If all mumbles are dimbles, and all thurbles are dimbles, are all thurbles mumbles?"

If you're like me your first reaction may well be, "what @#£%!$*? kind of question is that?" But, please, bear with me and try to answer it before reading on. No, really. Give it a go. I'd also like you to think about the type of person you are: are you an Arts student, with a preference for literature, performing arts, languages and the like? Or are you a Sciences student, with a preference for maths, logic, scientific reasoning and so on?

"If all mumbles are dimbles, and all thurbles are dimbles, are all thurbles mumbles?"

Answer:
TRUE: All thurbles are mumbles.
FALSE: All thurbles are not mumbles.

I prefer:
The Arts: literature, languages, interpretation and nuance is my thing!
The Sciences: maths, science, logic, facts and reasoning is where I'm at!

Let's look at the question in a bit more detail. I picked this example for a specific reason: it's a clear indicator of a fundamentally broken, inadequate testing mechanism. The question doesn't test a person's verbal reasoning ability, it tests how well a person can guess what the question's author actually meant. Here's why.

If you look in the answers section of an IQ test, you'll find the supposedly correct answer to this question is "False, all thurbles are not mumbles". This response will immediately delight all the Arts students, and infuriate the Sciences students. When an Arts student reads this question this is what they see:

"If all mumbles are dimbles, and all thurbles are dimbles, are all thurbles mumbles?"
"If all leaves are green, and all cars are green, are all cars leaves?"

Well hey, that's pretty obvious, right? A leaf is green, a car is green, but cars and leaves are definitely not the same thing. No arguments, the answer is "False". But, just before we congratulate ourselves and post that MENSA application, let's see what the Sciences student sees when they read the same question:

"If all mumbles are dimbles, and all thurbles are dimbles, are all thurbles mumbles?"
"If A equals B, and C equals B, does A equal C?"
-or-
"If 5 + 5 = 10, and 6 + 4 = 10, does 5 + 5 = 6 + 4?"

So as far as a Sciences student is concerned the answer is "True", unless anyone wants to argue that the answer to 5 + 5 does not give the same result as 6 + 4. What we have here is an exceptionally poorly chosen question, variations on which are used in numerous ability tests.

Apart from being deeply irritating, what does all that have to do with security? Let's look at the same question format from an IT security consultant's point of view. IT security falls firmly in the Sciences camp - to do the job you need an analytical mind and a logical approach, so we can probably guess the answer to the question before even reading it, right?

"If all mumbles are dimbles, and all thurbles are dimbles, are all thurbles mumbles?"
"If all serious security vulnerabilities are risks, and all critical problems are risks, are all serious security vulnerabilities critical problems?"

Again, on face value, the scientific answer is correct: "yes, A = B, C = B, C = A, therefore all serious vulnerabilities are critical problems". Unfortunately it's the wrong answer. Not all vulnerabilities are critical problems, regardless of what the vulnerability actually is. Imagine Microsoft have released a security bulletin saying all versions of Windows are vulnerable to a remotely exploitable code execution flaw. This is pretty bad news, but if your Windows server never gets connected to a network it doesn't actually affect you. It's not a risk. If that server runs a piece of machinery that would cost your company £1million for every hour it's offline, the risk-reward equation says you shouldn't patch the flaw.

This article was a very long winded way of showing how IT security specialists need to be able to flex their thought process. We operate in a scientific world where strict logical thinking is essential, but to properly assess risk and reward we have to change to a "fluffier" model that allows for a rounded view. Blindly patching every vulnerability and fixing every flaw might sound like a good move but the skilled security consultant knows that doesn't fly in the real, business world.

So, the next time you hear someone say "it's a security issue, we have to fix it", ask about the risk, not about the issue.

Submit to Digg      Save to del.icio.us      StumbleUpon