Scott raises a good question about whether the offered load in the
Data Comm test reflected "real world" conditions. The test application
(which, by the way, *is* available for use in contract testing) makes
thousands of connections, not hundreds, and it ran at Ethernet rather
than T1 rates.
I'm hearing more and more that the level at which we tested is, if
anything, a little light in terms of what Data Comm's readers see.
Remember, Data Comm's core audience is managers and designers of large
corporate Internet/Intranet installations. I've received numerous
inquiries about firewalls for FDDI, fast Ethernet, and ATM networks,
and the loads handled by such networks are far higher than anything we
modeled in our tests.
The only way to know for sure if a firewall can perform at an
appropriate level is to test it with the actual applications it'll
handle on the actual network where it'll reside. Marcus Ranum has
proposed an excellent performance modelling method that can be scaled
to any size load on any speed network. The URL is:
http://www.v-one.com/pubs/perf/approaches.htm
Regards
David Newman dnewman @
data .
com
Director, Data Comm Test Program voice 212-512-6182
Data Communications magazine fax 212-512-6833
I apologize for the implication that the Data Comm test is flawed. A more
accurate message should have been:
In my opinion, the Data Comm test *method* was/is slightly flawed, not the
test itself. The reason I make this claim is that I have seen hundreds of
simultaneous connections through a Raptor firewall with no problem at many
sites. Even though the Data Comm test did expose a bug in the Raptor
software (causing a connection to be dropped and the test not to complete
at that level), I would conclude that something in the test method was not
indicitive of a "real world" load since this result had never been seen
before by me or Raptor (they claim). Unfortunately, I have no further
evidence to back up my claim since as far as I know, the test code is
not publically available for inspection.
The bottom line is that Data Comm found a bug, Raptor fixed it and performs
very well in the retest, and Data Comm should be commended for shedding some
light on the murky world of firewall "performance".
scott
--
=====================================================================
Scott Bartram internet information services, inc.
email: scottb @
iis .
net 1680 East Gude Drive
voice: 301-340-1761 Rockville, MD 20850
|
|
