Forwarded message:
>From fc Sat Jun 24 09:31:16 1995
Subject: Re: Firewall performance, a last word... :)
To: mjr @
iwi .
com (Marcus J Ranum)
Date: Sat, 24 Jun 1995 09:31:16 -0400 (EDT)
In-Reply-To: <9506240305 .
AA15009 @
tis .
com> from "Marcus J Ranum" at Jun 23, 95 11:05:35 pm
X-Mailer: ELM [version 2.4 PL22]
Content-Type: text
Content-Length: 4369
> >There is nothing scientifically wrong with making a hypothesis, and
> >then gathering data to validate or disprove that hypothesis.
>
> Actually, I think there is.
>
> The order of operations is:
> 1) Gather data
> 2) Make a hypothesis that may explain the data
> 3) Derive a test for the hypothesis
> 4) Perform the test and gather more data
> 5) See if the initial data and data from the hypothetical
> test are consistent
> 6) See if the data supports the hypothesis
> 7) Post results to firewalls @
greatcircle :)
Actually, science is precisely the art of learning from mistakes (i.e.,
from refutations of theory by experiment).
Nothing I am aware of requires (1) except of course that without
imperical basis, hypotheses are mere speculations - but hypotheses
nonetheless.
(2) has been fiercely debated of the years - do our theories
explain or predict? Recently, explain has been all the fad, however,
prediction is the thing that makes testing possible by doing experiments
to find refutations (or less interestingly confirmations).
(3) is a test in the sense of trying to generate a refutation -
most of us test in a different way - by trying to confirm proper
operation in stead of by trying to find failures. Furthermore, tests
are rarely "derived" - they are one of the most interesting
non-systematic things scientists create.
(4), (5), and (6) are commonly put under the single heading of
doing the test - often even included in (3).
(7) is real comodic - but you seem to miss the real point of the
scientific method - the idea is that, if refutations are found, we
create new hypotheses. This is what makes the feedback system of science
work.
> Most of the time when people post here about firewall
> performance, the order of operations appears to be:
> 1) Formulate a hypothesis
> 2) Announce it as "likely" on firewalls @
greatcircle
> 3) Get jumped all over by mjr :)
>
> This discussion has gotten rather far afield, since it's
> really about computer science, not firewalls. After all, it'd only
> be about firewalls if someone were posting some test results and
> methodologies relating to firewalls. :)
Computer "science" as they call it is not really a science at
all. It is almost entirely a combination of mathematics (couched as
science) and philosophy. Few CS papers formulate hypotheses or do the
rest of what science requires. CS papers primarily posit assumptions
and prove theorums - or claim philosophies and expound upon them. (send
malicious replies to threatenning-letters @
whitehouse .
gov)
So to get back afield, the issue of the tradeoff between
security and performance has been around for a long time, and it has
rarely been resolved to the point of a number. But I have one from a
system designed for some unknown DoD group with extreme integrity,
availability, and confidentiality requirements. The penalty was about
40% for protection. That is, of 100% of the total performance of the
machine, 40% went to protection, and the remaining 60% went to doing
whatever else the machine did.
I am interested in devising a simple, but thorough, test for the
secure W3 server provided on this server. Perhaps a good way to look at
the issue of testing is to, as a group, consider the issues in testing
this relatively small and easily understood program. I have often
found that by starting with simple examples such as this one, we can
learn about more complex issues of testing big things like firewalls.
I will start the ball rolling by noticing that a complete test
is infeasible because the number of possible states and inputs is far
too large to complete such a test in the expected lifetime of the
universe - especially at 14,400 baud. If there is genuine interest, I
will set up a special test socket and bring up a test area on my server
to facilitate the process.
--
-> See: Info-Sec Heaven using our New Super Secure World-Wide-Web Server
-> Free: Test your system's security (scans deeper than SATAN or ISS!)
---------------------- both at URL: http://all.net ----------------------
-> Read: "Protection and Security on the Information Superhighway"
John Wiley and Sons, 1995 ISBN 0-471-11389-1, 320 pp, $24.95
-------------------------------------------------------------------------
Management Analytics - 216-686-0090 - PO Box 1480, Hudson, OH 44236
--
-> See: Info-Sec Heaven using our New Super Secure World-Wide-Web Server
-> Free: Test your system's security (scans deeper than SATAN or ISS!)
---------------------- both at URL: http://all.net ----------------------
-> Read: "Protection and Security on the Information Superhighway"
John Wiley and Sons, 1995 ISBN 0-471-11389-1, 320 pp, $24.95
-------------------------------------------------------------------------
Management Analytics - 216-686-0090 - PO Box 1480, Hudson, OH 44236
Follow-Ups:
|
|
