Home
Magazine
News
Latest News
Press Release Centre
News Archive
Companies
Jobs
Events
About us
Contact

Follow us:







Test of the century

28 September 2014 3:00
By Edward Bishop

Procedure

(Thanks to The Register: see http://theregister.co.uk/2014/09/24/bash_shell_vuln retrieved 28th Sept 2014 0300hrs UTC).

Execute

env X="() { :;} ; echo busted" /bin/sh -c "echo completed"

Expected outcome

what is returned DOES NOT include the string

busted

What makes this test the best

The defect this test detects is, to general knowledge, the worst ever, and it has been in very widespread production for 22 years. The test crackles with danger, as the world is now seeing.

I think the test is very beautiful in several aspects, and especially in its simplicity; but these are not important compared to the harm it will cause.

How testing could have helped

Heartbleed was detected by PT contributor and advertiser Codenomicon, using its advanced fuzzing tools. It is almost as surprising as Shellshock, although less simple. Static analysis vendor and PT issue sponsor Coverity added, quickly, detection to its algorithms. That proves that those algorithms could have been extended to detect it first. My guess is Coverity was unlucky and it could approximately equally as likely have been the other way around. A statistician with sufficient understanding of both organizationís methods could calculate the likelihood of my being right. Unfortunately I donít have time.

In both cases, the test strategy should (and may) have aimed to execute the test earlier. How could testing tactics have stood the best chance of achieving that? Any empirical or code-based test approach depends to a large extent on luck. Often, luck is in and important defects are found.

But not often enough. Shellshock proves that. It reinforces that no amount of expensive empirical testing can replace essential analytical testing.

Specifying, formally, Bashís parse algorithm then applying basic test analysis fed back to that specification would have prevented the defect. That inexpensive procedure should ideally have been done before the parser was coded, but could have been done at any time and would have saved harm by enabling the fix to be applied quietly.

I intend no criticism of the community that has developed Bash and given it to the world. It is not obliged to apply formal analytical testing. But if only someone had.



Related stories
The year ahead
TestEXPO - testing times ahead
Itís time to dream at TestExpo 2015
PT on hiatus
Social media is very defective


Up


Copyright © 2004-2017
Professional Tester Inc.
All rights reserved.
Legal Information.