On Sun, 22 Feb 1998, Paul D. Robertson wrote:
> On Sun, 22 Feb 1998, Bennett Todd wrote:
> > 1998-02-21-21:44:09 Paul D. Robertson:
> > > I honestly don't think you can provide a complete network security audit
> > > without having been in the trenches dealing with the issues for a few
> > > *years*.
> > That's dead-on perfect. When we demanded skilled auditors and insisted
> > on interviewing them to determine that they really knew enough, the big
> > firms fielded teams that included consultants they'd pulled in specially
> > for our audit --- and those consultants were experienced security admins
> > from large sites.
> > _That_ is a credential that counts: someone who successfully maintains
> > the security at a large site knows enough to test someone else's
> > security implementation.
> This ties back in, and I'm interested in it a great deal. Were those
> folks who were pulled in certified, and were the original team you were
> disatisfied also certified? In other words, was there a way to seperate
> the wheat from the chaff without being able to personally identify the wheat?
Yes, this ties the thread back together, and to some of the points I have
been trying to raise is how to separate out the *real* good people with
not only the technical background but also the ability to manage a
security audit and product a high level report to those upper management
type people and a low level detailed report to the sysadmins or IS/IT
management teams to address the results of the report.
The other end of this thread is how to properly disseminate the knowledge
among the people with 'CLUE' to the people with a growing desire to get a
'CLUE' and the experience to backup it up with. The Big N-1 firms have
jumped in recently gathered up a whole bunch of people with a 'CLUE' and
put together a team of wannabes with the people who have been in the
trenches for a while to further extend their service line. In the
client's mind their is perceived perception that the people doing the work
actually have been trained to conduct a security audit when they are still
learning. As Todd Bennett points out he screens the firm's candidates,
backgrounds, skill sets for the particular security audit in question. I
wish all customers of the Big N-1 did the same to avoid problems during
the course of the work that is being performed.
> How do we seperate the guys who can only hit the 10 ring at the range from
> those who've been in the trenches ducking real bullets and survived?
There are some are still in the trenches learning and documenting their
findings into dandy little troubleshooting tutorials and presenting them
at various conferences.
> Moreover, given the legal climate that can't discuss dismissal, how do we know
> that we're not looking at a ghost that's been dead a while?
I guess it just a roll of the dice and you try to screen the people are
very active in the security auditing field and assisting the security
industry suggesting improvement and such.
> Is it that this industry needs a way to periodicly free up the currently
> practicing for peer reviews? Does that make more sense than certifying
> "those who would be" practicioners? Certainly it would make the process
> of finding "those who would be" easier, since they'd be knocking down the
> doors to get in. That would give a rationale for having such a process
> which might even be palatable for businesses to free up their own
> temporarily. Maybe we could go from the Fortune 500 to the Firewall 500?
The security industry as I have stated before is still in its infancy and
definitely needs to be periodically cleaned. A peer review would
definitely assist some of the larger type organizations to help find their
To carry your thoughts one further, a consumer report type review of the
Big N-1 firms on their security auditing business practice and their
people would definitely alleviate some of the screening that Todd Bennett
alludes to in his above statements. Quarterly audits of the companies to
ensure that no wrongdoing and that proper and up to date training would be
A couple posters to this thread have suggested putting together an
Auditing Methodology and Practice forum to discuss creating standards in
security auditing and recommendations on courses, books, etc..
"Original developer and researcher for Firewall Troubleshooting Tutorial"
> > Thinking out loud,
> Paul D. Robertson "My statements in this message are personal opinions
> proberts @
net which may have no basis whatsoever in fact."