1998-02-16-15:58:57 Paul D. Robertson:
> 1998-02-16 Bennett Todd wrote:
> > If you're getting a systems security audit, then the results should
> > teach you something. If they don't then whoever paid for it was wasting
> > their money.
> Not necessarily, there are times when you're doing all you can with what
> you have, and the point of the audit is to ensure that you're "playing by
> the rules" and doing all you can with what you have.
Being told by an outsider, after independant examination, that you're
doing the best that can be done to achieve your documented security
goals --- that's worth a bunch! I've only had that happen once. It's a
real treat. Plus, getting an outside auditor to endorse goals and
strategies can be a powerful reinforcement; for instance, they can throw
their weight behind a major OS upgrade when some people are stalling.
> > I speak from recent personal experience. We had one audit come roaring
> > through, complete waste of time, [...]
> Not a complete waste if you updated your auditing requirements, no?
Good point! Somehow I hadn't managed to think about it that way,
becore...but yup, if you succeed in learning from mistakes then they are
no longer a _complete_ waste of time. Might not be the shortest and most
efficient route to get where you're going, but at least you can turn 'em
into progress after a fashion.
> > Next time and audit came around, we insisted on interviewing the people
> > who would be auditing us, grilling them to make sure they understood the
> > tools we were using to implement our security structure. Our stance was
> > that unless they knew enough to be able to understand our security
> > implementation, they couldn't analyze it for flaws and criticise it
> > inteligently. Worked like a champ.
> How many interviewees did you have to go through for this?
We had protracted discussions with teams fielded by two major auditing
firms --- and they knew what we were attempting to do up front, when
they were proposing their teams. Either one of them could have done a
decent job, we picked the one that seemed to have more depth in our
technology. But just by insisting that we had to be able to learn from
our auditors, and insisting on interviewing them to check this point, we
got a whole different grade of people.
> > The rule is simple: don't waste money getting audited by people who know
> > less than you do.
> That doesn't leave you with a very big audit pool...
Not a huge one, no, but big enough. They don't have to know more than
you do about every detail; they need to know enough to give a reasonable
overall examination and evaluation of your setup, and they need to have
enough depth so they'll know some useful details you don't.