Darn the SOX, We Need More Security Ahead
The PCAOB is introducing new guidance to help lower the overall cost and, presumably, increase the effectiveness of SOX 404 audits. It needs to use this opportunity to help fix some root causes, not just tell us how to find more symptoms.
This past December, the PCAOB announced that it would propose for public comment a “new standard on auditing internal control…designed to focus the auditor on the most important matters, increasing the likelihood that material weaknesses will be found…” The proposal itself can be found at http://www.pcaobus.org/Rules/Docket_021/2006-12-19_Release_No._2006-007.pdf.
Starting on page 93 of the document, there is a section on “Benchmarking of Automated Controls.” It includes guidance like “Entirely automated application controls are generally not subject to breakdowns due to human failure,” which is clearly not true since bad input makes functional application security controls fail all the time. Telling auditors that automated application controls automatically get a gold star is not a step in the right direction.
It does go on to suggest that the benchmarking strategy take into account the importance of the effect of related files, tables, data, and parameters on the consistent and effective functioning of the automated application. That’s a good thing.
The document then suggests that “If general controls over program changes, access to programs, and computer operations are effective and continue to be tested, and if the auditor verifies that the automated application control has not changed since the auditor established a baseline (i.e., last tested the application control), the auditor may conclude that the automated application control continues to be effective without repeating the prior year’s specific tests of the operation of the automated application control.”
So, never mind that new attacks are discovered with unnerving frequency and that perfectly good code can suddenly become not so great (think crypto algorithms), the apparent recommendation is that if you didn’t or couldn’t poke hard enough to break it over some time period, then it’s okay to skip it later. Approaches like this where we’re considering only functional changes and not testing skill and depth can’t be effective.
How many times has someone walked up and spotted a problem you failed to notice time and time again. As organizations periodically change auditing firms, expect huge increases in reported problems.
The proposed guidance gives the following factors to use when deciding to use benchmarking:
- The extent to which the application control can be matched to a defined program within an application;
- The extent to which the application is stable (i.e., there are few changes from period to period); and
- The availability and reliability of a report of the compilation dates of the programs placed in production. (This information may be used as evidence that controls within the program have not changed.)
This wording is still neglecting changes on the threat side of the equation. Just as many castles were considered impregnable until about five seconds before the first cannonball hit, many lines of code were considered secure right up to the point where the breach story appeared in the newspaper.
The guidance gives the following factors to use when deciding whether to reestablish the benchmarking baseline:
- The effectiveness of the IT control environment, including controls over application and system software acquisition and maintenance, access controls and computer operations;
- The auditor’s understanding of the nature of changes, if any, on the specific programs that contain the controls;
- The nature and timing of other related tests;
- The consequences of errors associated with the application control that was benchmarked; and
- Whether the control is sensitive to other business factors that may have changed. For example, an automated control may have been designed with the assumption that only positive amounts will exist in a file. Such a control would no longer be effective if negative amounts (credits) begin to be posted to the account.
Okay, so if I know that most auditors and organizations use COSO as a governance model, and COBIT 4 to interpret COSO to arrive some IT control objectives, and I consider AI2, Acquire and maintain application software, important to my environment, then I might understand what “controls over application and system software acquisition and maintenance” means above. And, I might even read COBIT and see AI2.4, Application Security and Availability, which states “Address application security and availability requirements in response to identified risks, in line with data classification, the organization’s information security architecture and risk profile. Issues to consider include access rights and privilege management, protection of sensitive information at all stages, authentication and transaction integrity, and automatic recovery.” Oh wait, I only need security software, not software security. Sigh. Another opportunity missed.
There is a need here for any words that introduce something like the following:
The customer may have written their own software that is directly used in the financial reporting process and you, the auditor, should be aware not only of the software’s functional controls (e.g., I&A, encryption, entitlements), but you must also accrue confidence that their internal development practices and testing are sufficient to produce quality software that has at least some capability to protect itself from attack even in the event of catastrophic failures other general and IT security controls being considered as part of this SOX audit. While security software may comprise the majority of IT security controls, software security is the property that gives us confidence in their continued successful operation even in the certainty of ongoing attack. You must use a risk-based approach to accruing confidence, focusing on relevant factors that have a material effect on the software’s ability to meet business objectives even in an overtly hostile environment.
Now we’re thinking more about the problem and less about the symptoms.
Technorati Tags: software security
March 12th, 2007 at 3:54 pm
Although I agree with you that SOX could be improved in many ways, I also think that it has done plenty to move the software security ball down the field.
I even wrote it down in my darkreading column:
http://www.darkreading.com/document.asp?doc_id=119163
gem
March 12th, 2007 at 11:33 pm
Well, I’ll be pedantic for a moment, and not just because I enjoy it.
“SOX” is just this somewhat vague law out there that set the requirement for the creation of the PCAOB, who then worked with a lot of folks to create Auditing Standard #2, which was then interpreted as necessary by a buncha more folks for conducting audits of public firms. These are the feet on the street who actually moved the ball.
The reason why organizations, kicking and screaming, allowed these people to move the ball is very simple. SOX includes three great things: 1) organizations have to evaluate and disclose the effectiveness of their internal controls related to financial reporting; 2) independent auditors must attest that the disclosure is accurate; and, for lagniappe, 3) civil and criminal penalties.
This has resulted in billions of dollars of activity, but, in my humble opinion, only scant millions of dollars of progress spread out over two million public companies. Companies have spent much time in introspection to discover things about themselves they should have known all along. I feel only a few visionaries have used SOX to springboard their organization to a new maturity level.
–Sammy.
March 14th, 2007 at 1:21 pm
I think that most “business” people used SOX as a weapon of war: tired of all the uppity, all-too-rational IT types horning in on their territory, the “business” people managed to bring in SOX consultants who composed draconian anti-progress rules about computers, software, etc.
The move to “offshoring”, especially to India, is also a manifestation of this. Note that the Ideal Offshoring is done to India, with its long history of authoritarian culture, rather than to Russia, Romania, etc, where there is a long history of intellectualism.
I’m told that SOX per se doesn’t require the kind of punitive, committee-review-driven, top-down, Stalinist 5-year-plan kind of rules that almost always get put in place. So, I have to believe that the “business” people want to enforce their absolute control, and marginalize and pigeon-hole any IT types.
It’s all too easy to get mid-level management types to focus on a checklist of items. Hey, look! A shiny checklist! that makes the IT departments, and individuals in those departments, worth far less to the organization than they might otherwise, and also make them less of a threat to the “business” people in the old guy network.
Checklists are a lot like GUIs: they have no looping, no if-then-else and no conditionals, so they can’t describe large categories of behaviors and actions. But they’re also eminently understandable by the Borderline Personality Disorder suffers who typically inhabit the ranks of middle management.
March 20th, 2007 at 3:26 pm
Just to reinforce my assertion above that “business” people use checklists (a.k.a. Received Wisom, a.k.a. Dogma) as a weapon, take a gander at this blog entry: http://securitybuddha.com/2007/03/20/executive-sponsorship-and-commitment/
In this war story, a highly-placed cricket fan was able to get a corporate content filter removed because it caught “Middlesex” and “Sussex”, the English towns in which major Cricket clubs reside. “After this event information security became somewhat irrelevant for a period with hard nosed business folks using dogma to get their own way. The details of who is right and what really happened is largely irrelevant.” Same as with SOX “mandares” and “controls”.