Corey wrote:
> To all:
>
> I think some important important questions need to asked:
>
> 1. Who appointed the NCSA as the proper body to approve firewalls?
>
> 2. Do people realize that in order to be approved, a vendor must be a
> member of the NCSA?
>
> 3. Do people realize that the first vendors approved were all members
of
> the NCSA and as such got a timing advantage over other non-members?
>
> 4. Is it fair that all vendors, irrespective of size, must first pay a
> $22,000 membership fee?
>
> 5. Will the NCSA put a footnote on their "approved" list that only
those
> vendors willing to pay $22,000 have received the NCSA's approval?
>
> 6. Doesn't the "bundled" concept of membership and qualification for
> approval render whole process meaningless?
>
> 7. Have any members of NCSA not been approved?
>
> 8. What is NCSA doing with the funds received by its members? Is NCSA
a
> non-profit organization?
>
> I believe all of these questions need to be addressed before the NCSA
holds
> itdself out as the self-appointed arbiter of firewall quality assurance.
>
> Just one man's opinion
>
All very good questions.
As most subscribers to this list will be well aware, we were not short of
evaluation bodies in the first place.
So far no one has come up with a perfect evaluation system and probably
never will, so it comes down to deciding what risks each buyer is prepared
to take.
TCSEC/'Orange Book'
NCSC still evaluates product in the national (US) interest. The evaluation
has been free to the vendor but its still cost a great deal of money. The
vendor has to hire a VSA who has passed the NCSC VSA training and
examination system. Considerable work has to be done during an evaluation
to provide the system (hardware and software) and deal with the questions
and give the presentations necessary to support the NCSC evaluators.
The benefit of an NCSC certificate is that NCSC does not evaluate in the
vendors' interests and are a government controlled and funded agency
specifically established to be independent of vendor interest.
The risks of NCSC evaluation are several:
1. The process is slow and this means that the product is becoming
obsolete by the time the evaluation is complete.
2. The RAMP (rating maintenance programme) is also slow and
cumbersome so that the product available for delivery with a certificate
is much older than the latest version in vendor development.
3. The evaluation primarily covers assurance and not integrity or
availability.
4. The evaluation and certificate covers a system down to fine detail
like printer cables and much of that hardware will no longer be standard
production by the time the evaluation is complete.
5. The vendor jacks up the product price to reflect the cost of
development and evaluation support and because the product enjoys some
monopoly or quazi monopoly status through rarity of certificates.
6. TCSEC uses an incorrect model for the development processes
employed by vendors.
7. Rainbow Series is based strongly on Mil-Std 2167A which assumes a
detailed customer specification and custom engineering to meet that
specification.
8. The system doesnt allow for sub-system certification other than
you can have an evaluation which results in a D level ticket which is also
issued to failed products.
9. Even today an NCSC evaluated product may not be available to all
users, even inside the US. End-user certificates may still be required
before legal shipment.
ITSEC
European Governments recognised the weaknesses and strengths of the US
process and 4 countries worked together to produce ITSEC.
ITSEC has several benefits over the US NCSC system:
1. Any number of Commercial Licensed Evaluation Facilities can be
licensed. The UK ITSEC Scheme Body has already licensed 8 CLEFs (2 are US
owned subsidiaries). The German ITSEC Scheme Body is planning to license
additional CLEFs, possibly up to 120. France is planning to introduce a
CLEF system with somewhere between the UK and German licensing numbers.
That removes a major delay cause present in the US system where NCSC just
doesnt have the manpower to handle even the relatively small number of
products in the queue.
2. ITSEC certificates in the UK and Germany are mutually recognised
by an agreement between the 2 national schemes and other countries are due
to sign agreements this year in Europe and other areas.
3. Any vendor can present product for evaluation.
4. Any user can buy certified product - not just specialised
government agencies.
5. ITSEC measures Integrity and Availability as well as measuring
Assurance.
6. Software testing can be generic. Therefore a firewall mounted on
an Intel-based platform and a specific trusted OS can be certified as
meeting a particular TOE on any Intel platform which has a certified OS.
ITSEC also has risks:
1. The CLEFs do not issue licenses, only evaluation reports.
Certification is by the government run national ITSEC Scheme Bodies.
Therefore the system is only as good as the policing by the Scheme Bodies
who are able to place export and distribution controls on some products.
2. Although ITSEC is significantly faster than the NCSC system, its
still slow and still leads to obsolete product.
3. Generic platform certificates for software do introduce risk
because clone hardware may have vulnerabilities which were not present in
the model and manufacture of the hardware supplied as a base for
evaluation (this also applies to any platform component like the OS).
4. CLEF evaluation fees can be extortionate. ***BEFORE a CLEF objects
to that statement, I would qualify it. A small product which takes one
month to develop and document for evaluation can take a year to evaluate
at a high day rate. Charges are proportionately more realistic as the
product complexity increases.
5. There is not an established formal RAMP system and review of new
versions can be erratic across a number of products.
6. Some vendors with very good products cannot justify evaluation
costs and therefore a certified product is not necessarily the best
solution available. Thats particularly true as long as ITSEC evaluations
are in Europe and much product development is somewhere else. A vendor
(for example a US vendor) who has perhaps 20 years experience of providing
trusted solutions and who has already had successfull NCSC evaluations
still has to undergo a development assurance inspection. If he happens to
be based in a pleasant geographic area, some CLEFs may feel that its
necessary to send a small team over for several weeks to review design
processes, staying in the best hotels and charging a high day rate. This
is still not a proportionally high cost, provided that the vendor is
submitting many products over a period, because the inspection is for once
only. That may be another risk because the vendor might not employ the
same methods later on.
Common Criteria
This has yet to go into full operation and so the effects are potentially
unknown. However, it is based heavily on ITSEC so it is reasonable to
expect similar benefits and risks.
Provided many countries sign mutual acceptance and evaluation agreements
like the ITSEC agreements, CC will really become the international system.
The main risk may then be that not every Scheme Body may really work as
agreed and national interests may intrude. ITSEC for example lost several
benefits and introduced extra risk because Europe tried very hard to
accomodate the national interests of other countries in an attempt to
develop a true ISO style international criteria.
Self Certification.
Vendors will offer self certified and 'designed to meet' products. Some
vendors may be very correct and open while others will offer only a
marketing view of product achievements.
Potentially this is very high risk unless the vendor claims are nailed
down firmly in the procurement document and you can afford to take them
through the courts if necessary. Even then its still risky.
Self Evaluation with Test Suites
US NIST and UK NPL have both offered dial-in test suite facilities
including C2 test suites. I dont believe either service has attracted many
vendors and probably the benefits over self certification are minimal.
There are a growing number of test suites available for network security
and some of these could be used by vendors and customers alike. That does
of course assume that both are capable of driving them and being hoest
about the results. There is also the question of how up to date and
effective the test suites are. Like penetration testing, the result may
mean something or nothing.
Reality of systems
In the end, product testing is only a small part of the total equation in
risk management.
When TCSEC and ITSEC were established, there were two goals:
1. Force vendors to present adequately documented product with
identified Security Targets.
2. Make project procurement easier by removing some unnecessary
risks.
Neither system is a destination for governments, the destination is
accreditation and enforcement.
Unfortunately, buying certified product to build a system doesnt mean the
system achieves the same level, and vulnerabilities can be introduced
during integration and implementation.
Accreditation doesnt work unless you first produce a detailed risk policy
to provide something against which you can measure vulnerabilities and
decide which vulnerabilities have to be removed and which are acceptable
risks for YOUR business.
Accreditation means very little unless you have the means to enforce your
risk policy.
There are no short cuts.
As far as NCSA Certification goes, only experience will show if its worth
anything. Any trade club is vulnerable to vested interest and all the
certificate might show is that a particular vendor has paid the membership
fee.
OTOH it can be a huge benefit to the first vendors to join because it
provides their marketeers with something else to hype their products. That
advantage reduces as more vendors join the club and might become worthless
if some products with certificates are shown later to have severe
vulnerabilities.
Ian J-B.
|
|
