From network-automation-owner@greatcircle.com Thu Apr 7 13:32:53 2005 X-Original-To: network-automation@greatcircle.com Received: from [66.92.48.19] (localhost [127.0.0.1]) by mycroft.greatcircle.com (Postfix) with ESMTP id DE96332C69A for ; Thu, 7 Apr 2005 13:32:52 -0700 (PDT) Mime-Version: 1.0 Message-Id: Date: Thu, 7 Apr 2005 13:32:49 -0800 To: network-automation@greatcircle.com From: Brent Chapman Subject: Test to new Network Automation list Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Archive-Number: 200504/1 X-Sequence-Number: 1 This is a test message to the newly created Network-Automation@GreatCircle.COM mailing list. -Brent -- Brent Chapman -- Great Circle Associates, Inc. Specializing in network infrastructure for Silicon Valley since 1989 For info about us and our services, please see http://www.greatcircle.com/ Network Automation blog: http://www.greatcircle.com/blog/network_automation From network-automation-owner@greatcircle.com Fri Apr 8 13:21:56 2005 X-Original-To: network-automation@greatcircle.com Received: from darksun.binsh.com (darksun.doomathon.com [64.232.216.23]) by mycroft.greatcircle.com (Postfix) with ESMTP id 5A1EC32C222 for ; Fri, 8 Apr 2005 13:21:55 -0700 (PDT) Received: from darksun.binsh.com (localhost [127.0.0.1]) by darksun.binsh.com (8.12.6/8.12.6) with ESMTP id j38KMf5C026799 for ; Fri, 8 Apr 2005 13:22:41 -0700 (PDT) Received: from localhost (paxton@localhost) by darksun.binsh.com (8.12.6/8.12.6/Submit) with ESMTP id j38KMf2Z026796 for ; Fri, 8 Apr 2005 13:22:41 -0700 (PDT) X-Authentication-Warning: darksun.binsh.com: paxton owned process doing -bs Date: Fri, 8 Apr 2005 13:22:41 -0700 (PDT) From: Paxton To: Subject: available network automation tools Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Archive-Number: 200504/2 X-Sequence-Number: 2 I haven't seen any messages yet so I have no idea who has signed up, but I would love to hear a frank discussion of currently available network automation/configuration management (not monitoring) tools, both $$$ and open source. I was interested by Brent's original NANOG posting, in which he states: ..."I'm not simply talking about [..] device configuration monitoring systems [..] Instead I'm talking about systems that will start from a description of how a network ought to be configured, then interact with the various devices on that network to make it so..." What currently available tools actually do this? IMO the $$$ tools out there today (or at least those I have seen referenced, ie the network world article, rendition/Opsware, etc) all are pandering to the sarbanes-oxley scare tactics because that's where the money is. If you really look at them and peel away the marketing fluff and hand-waving, they are all basically a configuration monitoring systems with a stamp on the cd that says: your sarbanes-oxley problems solved here! Is their goal even to solve network automation problems? I find it ironically humorous that rendition renamed true control to "network automation". Maybe they should have renamed it: Sarbanes-Oxley BandAid. Not to pick on rendition, but the reality is there's money in checkbox sarbanes-oxley solutions - and that's money in the right place (execs) and a lot of it. So are real network management solutions getting left in the dust? And what's worse at least IMO is that these guys all claim to provide network management solutions, but don't actually provide value to network administrators (or that isn't their main goal, its an afterthought if its given any consideration at all), and because the money is already spent, the network administrators don't get tools that might actually solve a real problem. I hope this is just controversial enough to spur on some conversation, because I would really like to hear everyone else's opinions and experiences, and what else is out there that I haven't seen yet. Can anyone recommend open source tools for configuration management/network automation, or is there interest in starting such a project? Thanks! From network-automation-owner@greatcircle.com Fri Apr 8 13:42:00 2005 X-Original-To: Network-Automation@GreatCircle.COM Received: from [66.92.48.19] (localhost [127.0.0.1]) by mycroft.greatcircle.com (Postfix) with ESMTP id E7CD032C29F for ; Fri, 8 Apr 2005 13:41:59 -0700 (PDT) Mime-Version: 1.0 Message-Id: Date: Fri, 8 Apr 2005 13:41:57 -0700 To: Network-Automation@GreatCircle.COM From: Brent Chapman Subject: Welcome, and BoF at USENIX next week Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Archive-Number: 200504/3 X-Sequence-Number: 3 It's been less than 24 hours since I announced this list, and we've already passed the 200 subscriber mark... Welcome one and all! I'm looking forward to some very interesting discussions on this list. If any of you are going to be at the USENIX conference in Anaheim next week, I've scheduled a Network Automation BoF (Birds of a Feather session, where folks interested in a particular topic get together to chat about it) for Wednesday night, 8:30-9:30pm. Right now, they've got us scheduled in Salon H, but that's subject to change, so check the scheduling board at the conference. BoF info: Automating Network Configuration & Management Organizer/Moderator: Brent Chapman, Great Circle Associates Wednesday, April 13, 8:30 pm-9:30 pm, Salon H What's the state of the art for automated network configuration and management? What systems and tools are available, either freely or commercially? Where are these issues being considered and discussed? Over the last 15 years or so, much of the research in the system administration field has focused on automation. It's now well accepted that a well-run operation doesn't manage 10,000 servers individually, but rather uses tools like cfengine to manage definitions of those servers and then create instances of those servers as needed. In the networking world, though, most of us seem to be still manually configuring (and reconfiguring) every device. Some URLs: All scheduled BoF: http://www.usenix.org/events/usenix05/bofs.html Our BoF: http://www.usenix.org/events/usenix05/bofs.html#automating Conference info: http://www.usenix.org/events/usenix05/ So, now that we're all here, let the games begin! ;-) -Brent -- Brent Chapman -- Great Circle Associates, Inc. Specializing in network infrastructure for Silicon Valley since 1989 For info about us and our services, please see http://www.greatcircle.com/ Network Automation blog: http://www.greatcircle.com/blog/network_automation From network-automation-owner@greatcircle.com Fri Apr 8 13:56:58 2005 X-Original-To: network-automation@greatcircle.com Received: from iad2smtpout03.target.com (smtpout03.target.com [161.225.129.10]) by mycroft.greatcircle.com (Postfix) with ESMTP id D8FC132C2A7 for ; Fri, 8 Apr 2005 13:56:57 -0700 (PDT) Received: from emailhub02.target.com ([10.114.73.22]) by iad2smtpout03.target.com with Microsoft SMTPSVC(6.0.3790.211); Fri, 8 Apr 2005 15:56:57 -0500 Received: from EMAILSTORE13.target.com ([10.114.81.32]) by emailhub02.target.com with Microsoft SMTPSVC(6.0.3790.211); Fri, 8 Apr 2005 15:56:59 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: Re: available network automation tools Date: Fri, 8 Apr 2005 15:56:55 -0500 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: available network automation tools Thread-Index: AcU8eKEz+8ZTeH3/QGWpUOt/S75iCgAANXdg From: "Network.Security" To: X-OriginalArrivalTime: 08 Apr 2005 20:56:59.0012 (UTC) FILETIME=[81703440:01C53C7D] X-Archive-Number: 200504/4 X-Sequence-Number: 4 OK, I'll bite. We use Rendition (now Opsware) for config mgmt. of our network stuff and the change detection / archival function has already saved my butt a number of times. Our engineers gripe about the Big-Brother aspect, but per the quasi-rant about SOX / CISP / PCI it's a fact of life now for any SEC-filing corp at a minimum. I realize this is somewhat OT for this list, but SOX (in general) shouldn't really matter to network admins (network meaning L1 to ~L4), as SOX is all about altering the financial data. As a network person, you can certainly see all of that data, but you can't change it (packet injection doesn't count, most apps pick up on that sort of activity even if they don't know why they know :) For that you need to be a server admin / DBA, so for true network people, we don't really care. (Loose generalization there. Re: caring) As for Opsware, their SOX report is static. It doesn't tell you anything. It's just text so the tool offers no value to that. Now, CISP / PCI on the other hand, that's the project funding behemoth you've all been waiting for. If you need money, say it's for PCI and poof-like-magic, here's the cash to make it happen. PCI has some fairly strict requirements that are defined to the network level regarding open ports, encryption schemes, use of clear-text, etc. Tools like Opsware can help enforce or at least notify on those data points. Re: free tools, I think we've all heard of RANCID as a config-o-monitor (I personally am CVS debilitated and have not yet been able to make it work on any platform). Big companies do not like free tools. That's why Linux was not making progress in large enterprises until we could start paying for it (aka Red Hat). If they can't get maintenance / support for something that the business needs, it's not coming into the environment. This makes sense, though not really from a "help us stay on the cutting-edge of technology aspect". Opsware and Voyence (we demo-ed them) both do some configuration templating so that if you are building out CPE, you can have it make your router configs or whatever, but for already installed networks, the templating is not valuable. I'm all about the configuration policy piece, I want to know how many of my devices don't have enough ntp servers configured, that sort of thing or down the road to make sure my QoS policies are consistent across the board. But that's all to get me to a point in time where everything is "right", it doesn't help me deploy new services all that differently than my perl scripts did before. What I think you are talking about is an application aware network provisioning system. Something that is aware of all possible topological paths between endpoints and is smart enough to know how to configure all the hops / connections in between the two to make something happen. Like a DOS mitigation system or punching holes through a series of firewalls or configuring multi-hop VPN tunnels. Yeah, that doesn't exist as far as I know. Or more to the point, I'm sure these tools could do that, but the work required on the front-end won't end up saving you anything on the back-end. Oh and it has to be vendor-agnostic. Heh. That market is more in the NetDoctor or related simulation style configuration analyzers that do the what-if type stuff or there are a couple other QoS policers out there that do something similar, but they are niche market tools for QoS only and appliances at that. So to your question, No would be my answer, there isn't something like that out there today, there are point solutions that solve particular aspects of the overall issue, but nothing end-to-end. BTW, Opsware and others would love to see this list if it grows and utilize it as a tool for soliciting industry-wide customer feedback. I would suggest consideration of that either for it or against (I don't really care) and noting it in the policy. Scott.altman@target.com From network-automation-owner@greatcircle.com Fri Apr 8 14:45:16 2005 X-Original-To: network-automation@greatcircle.com Received: from mail02inhq.keynote.com (mail02inhq.keynote.com [65.198.48.194]) by mycroft.greatcircle.com (Postfix) with ESMTP id 4460732C3A9 for ; Fri, 8 Apr 2005 14:18:18 -0700 (PDT) Received: from exchange04inhq.int.win.keynote.com (exchange04inhq.keynote.com [65.198.49.44]) by mail02inhq.keynote.com (8.12.11/8.12.11) with ESMTP id j38LIHjj024513 for ; Fri, 8 Apr 2005 14:18:18 -0700 X-Relayed-By: mail02inhq.keynote.com x-mimeole: Produced By Microsoft Exchange V6.0.6603.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C53C80.7B387BBB" Subject: ACL management Date: Fri, 8 Apr 2005 14:18:17 -0700 Message-ID: <663A9295DD2FFB46A85D0264EEB83F60CC144B@exchange04inhq.keynote.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: ACL management Thread-Index: AcU8gHsafKWCV076RgmleNYt+sERMA== From: "Alan Young" To: X-Archive-Number: 200504/5 X-Sequence-Number: 5 This is a multi-part message in MIME format. ------_=_NextPart_001_01C53C80.7B387BBB Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable All, =20 What products out there can I use to automate/simplify ACL management? Cur= rently, the process I follow is a very tedious and manual process. =20 I am specifically thinking of applications that work in a Cisco centric env= ironment. Before everyone suggests CiscoWorks ACL manager, I would like to= state that I have had many problems with CiscoWorks and would rather not u= se it. However, CiscoWorks ACL Manager might be a different beast, so I wo= uld like to hear people's thoughts on the product. =20 Having a easy way to make changes to an ACL would really help many people u= sing old bogon lists from places such as: http://www.cymru.com/Documents/se= cure-ios-template.html=20 =20 Thanks, =20 -Alan Young ------_=_NextPart_001_01C53C80.7B387BBB Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

All,

 <= /span>

What products out there can I use to automate/simplify ACL managemen= t?  Currently, the process I follow i= s a very tedious and manual process.

 <= /span>

I am specifically thinking of applications that work in a Cisco cent= ric environment.  Before everyone suggests CiscoWor= ks ACL manager, I would like to state that I have had many problems with CiscoWorks and would rather not use it.  However, CiscoWorks ACL Manager might be a different beast, so I wou= ld like to hear people’s thoughts on the product.

 <= /span>

Having a easy way to make changes to an ACL would really help many people using old bogon lists from places such as: http://www= .cymru.com/Documents/secure-ios-template.html

 <= /span>

Thanks,

 <= /span>

-Alan Young

------_=_NextPart_001_01C53C80.7B387BBB-- From network-automation-owner@greatcircle.com Fri Apr 8 15:43:48 2005 X-Original-To: network-automation@greatcircle.com Received: from mta1.lbl.gov (mta1.lbl.gov [128.3.41.24]) by mycroft.greatcircle.com (Postfix) with ESMTP id BEBDF32C340 for ; Fri, 8 Apr 2005 15:43:47 -0700 (PDT) Received: from mta1.lbl.gov (localhost [127.0.0.1]) by mta1.lbl.gov (8.12.10/8.12.10) with ESMTP id j38Mhin3023126 for ; Fri, 8 Apr 2005 15:43:44 -0700 (PDT) Received: from [127.0.0.1] (dhcp163-8.nersc.gov [128.55.8.163]) by mta1.lbl.gov (8.12.10/8.12.10) with ESMTP id j38MhhP9023119; Fri, 8 Apr 2005 15:43:43 -0700 (PDT) Message-ID: <4257091F.4050700@nersc.gov> Date: Fri, 08 Apr 2005 15:43:43 -0700 From: Eli Dart Reply-To: dart@nersc.gov Organization: NERSC Center, LBNL User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Alan Young Cc: network-automation@greatcircle.com Subject: Re: ACL management References: <663A9295DD2FFB46A85D0264EEB83F60CC144B@exchange04inhq.keynote.com> In-Reply-To: <663A9295DD2FFB46A85D0264EEB83F60CC144B@exchange04inhq.keynote.com> X-Enigmail-Version: 0.90.1.1 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Archive-Number: 200504/6 X-Sequence-Number: 6 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Alan Young wrote: > All, > > What products out there can I use to automate/simplify ACL management? For IOS-like boxes, we've found that a tftp server works very well. You edit the command file (which contains the set of commands to install the acl) on the tftp server (where you have version control, etc) and then pull it into the router via tftp when you're done editing it. If you have to push a given list out to many routers, I could envision a perl Net::Telnet script to install an acl bound to a given interface in a given direction. You could also put together a make infrastructure that created the acls and then your perl script would only pull them down to the appropriate router. I've not had the need to build such a beast, but it feels fairly straightforward. --eli -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (MingW32) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCVwkfLTFEeF+CsrMRAu4BAKCdgG2LIWLuPeLIhQYRO2v+ji03ZQCgzkpy YvQjwM4vCBI281tbMzph8Cw= =kBdI -----END PGP SIGNATURE----- From network-automation-owner@greatcircle.com Fri Apr 8 18:11:21 2005 X-Original-To: network-automation@greatcircle.com Received: from mail.axint.net (mail.axint.net [38.116.133.5]) by mycroft.greatcircle.com (Postfix) with ESMTP id 805DA32C213 for ; Fri, 8 Apr 2005 18:11:20 -0700 (PDT) Received: (qmail 18667 invoked by uid 514); 9 Apr 2005 01:11:18 -0000 Received: from 38.116.133.27 by mail.axint.net (envelope-from , uid 89) with qmail-scanner-1.24-st-qms (clamdscan: 0.83/741. bitdefender: v7.0/2490/101512. perlscan: 1.24-st-qms. Clear:RC:1(38.116.133.27):. Processed in 0.602935 secs); 09 Apr 2005 01:11:18 -0000 Received: from unknown (HELO cs.axint.net) (cstone@axint.net@38.116.133.27) by 0 with (RC4-MD5 encrypted) SMTP; 9 Apr 2005 01:11:18 -0000 From: Chris Stone Organization: AxisInternet, Inc. To: network-automation@greatcircle.com Subject: Re: ACL management Date: Fri, 8 Apr 2005 19:11:11 -0600 User-Agent: KMail/1.6.2 References: <663A9295DD2FFB46A85D0264EEB83F60CC144B@exchange04inhq.keynote.com> <4257091F.4050700@nersc.gov> In-Reply-To: <4257091F.4050700@nersc.gov> MIME-Version: 1.0 Content-Disposition: inline Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Message-Id: <200504081911.15864.cstone@axint.net> X-Archive-Number: 200504/7 X-Sequence-Number: 7 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Friday 08 April 2005 04:43 pm, Eli Dart wrote: > Alan Young wrote: > > What products out there can I use to automate/simplify ACL management? > > For IOS-like boxes, we've found that a tftp server works very well. > You edit the command file (which contains the set of commands to > install the acl) on the tftp server (where you have version control, > etc) and then pull it into the router via tftp when you're done > editing it. > > If you have to push a given list out to many routers, I could > envision a perl Net::Telnet script to install an acl bound to a > given interface in a given direction. You could also put together a > make infrastructure that created the acls and then your perl script > would only pull them down to the appropriate router. I've not had > the need to build such a beast, but it feels fairly straightforward. There is an open source project on SourceForge called the Cisco-centril Ope= n=20 Source Initiative that has a bunch of scripts for dealing with cisco router= s=20 and switches, include ACL management. Recommend a look: http://sourceforge.net/projects/cosi-nms/ Chris Stone, MCSE AxisInternet, Inc. www.axint.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFCVyuvG4PxJjbMvv0RAij1AKCeKR5s7Xhy1VCpgL6fYXTFzRBkWACgrQdd B5vSSaEhn+R2kkhzElnngSc=3D =3D4nWF -----END PGP SIGNATURE----- From network-automation-owner@greatcircle.com Fri Apr 8 18:53:54 2005 X-Original-To: network-automation@greatcircle.com Received: from mtxexch01.add0.masergy.com (unknown [64.47.104.21]) by mycroft.greatcircle.com (Postfix) with ESMTP id A261832C193 for ; Fri, 8 Apr 2005 18:53:53 -0700 (PDT) Received: from [64.47.15.110] ([64.47.15.110]) by mtxexch01.add0.masergy.com with Microsoft SMTPSVC(6.0.3790.211); Fri, 8 Apr 2005 20:53:11 -0500 Message-ID: <42573546.7050503@gmail.com> Date: Fri, 08 Apr 2005 21:52:06 -0400 From: Kirby Files User-Agent: Mozilla Thunderbird 1.0.1 (X11/20050309) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Paxton Cc: network-automation@greatcircle.com Subject: Re: available network automation tools References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 09 Apr 2005 01:53:11.0419 (UTC) FILETIME=[E29E4CB0:01C53CA6] X-Archive-Number: 200504/8 X-Sequence-Number: 8 Paxton wrote on 04/08/2005 04:22 PM: > I was interested by Brent's original NANOG posting, in which he states: > > ..."I'm not simply talking about [..] device configuration monitoring > systems [..] Instead I'm talking about systems that will start from a > description of how a network ought to be configured, then interact with > the various devices on that network to make it so..." > > What currently available tools actually do this? None that I've seen. Before going the route of building our own internal system for vendor-independent configuration generation and management, I met with every CM vendor I could find, including a whole bunch of dot-com busts. To name a few (the more promising ones): Orchestream Dorado Goldwire Digital Fairways Cplane Network Mantra Rendition Most of these focused on two things: config backup, and limited Cisco service config templating. What we were looking for was a full network equipment and relationship model, generic enough to model all types of routers, switches and containers, along with a scriptable configuration generation engine to interpret that model into device-dependent configuration deltas, and a service activation manager to resolve dependencies (which services required changes to which devices) and schedule config changes. Some of the tools above evolved to be support a little more than just Cisco, and some of the config templating got a little more sophisticated, but most have since died or been acquired, and none meet the above goals. Unfortunately, 4 years later, we still wouldn't be able to replace our homegrown configuration management system with any COTS system that had comparable features. And the importance of having a queryable network model cannot be overstated. This has become the core of our OSS systems: driving data collection, automating NMS configuration and performance measurement, and enabling much better inventory control than we ever had before. Originally, I had hoped that the DMTF Common Information Model and Directory Enabled Networking initiatives might yield some useful products in this direction, where TMN failed to help. But these took the same path of TMN of over-abstracting and over-specifying particularly at the physical layer, making them unworkable for logical network modeling. --kirby files NMS Software Lead Masergy Communications From network-automation-owner@greatcircle.com Sat Apr 9 08:05:53 2005 X-Original-To: network-automation@greatcircle.com Received: from smtp-auth.no-ip.com (smtp-auth.no-ip.com [8.4.112.95]) by mycroft.greatcircle.com (Postfix) with SMTP id 391C732C19D for ; Sat, 9 Apr 2005 08:05:51 -0700 (PDT) Received: (qmail 20461 invoked by uid 89); 9 Apr 2005 15:05:49 -0000 Received: from unknown (HELO localhost) (max.reid@saikonetworks.com@192.168.15.3) by mail1.no-ip.com with SMTP; 9 Apr 2005 15:05:49 -0000 Received: from wbar3.sea1-4-5-119-184.sea1.dsl-verizon.net (wbar3.sea1-4-5-119-184.sea1.dsl-verizon.net [4.5.119.184]) by webmail.no-ip.com (IMP) with HTTP for ; Sat, 9 Apr 2005 08:05:49 -0700 Message-ID: <1113059149.4257ef4d5e7aa@webmail.no-ip.com> Date: Sat, 9 Apr 2005 08:05:49 -0700 From: max.reid@saikonetworks.com To: Paxton Cc: network-automation@greatcircle.com Subject: Re: available network automation tools References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit User-Agent: Internet Messaging Program (IMP) 3.2.1 X-WebMail: No-IP.com X-Originating-IP: 4.5.119.184 X-Archive-Number: 200504/9 X-Sequence-Number: 9 Hi all, Currently, I'm pretty good at putting together stuff like subversion, rancid, RRDtool, Cacti, Nagios, tftp, scp, together into a somewhat useful package for basic stuff, automated diffs and version control and Monitoring. I think we can all agree that such homegrown systems are great for that sort of thing. There certainly isn't a cfengine type application for multivendor network devices, but with cisco Incorporating Tcl shells and precompilers in IOS, F5 using a Tcl shell to replace their proprietary scripting language, There just needs to be an Open Reference model or XML schema to describe device functions... I don't think we're there yet. What I don't like are some of the things various vendors are doing to try to address the problem in their devices, like the Cisco "AutoSecure" feature. I suppose this will open discussion into how much automation is a good thing. If it saves me from carpel tunnel or having to produce funky perl code, then perhaps it is useful. I think we get into trouble when the tools start to dynamically alter the network topology. If you take a look at some IDS/IDP systems, you'll understand what I mean. While they are policy driven, they aren't very useful out of the box and take considerable work in order to have them function properly for your specific environment. Regards, Max Quoting Paxton : > > I haven't seen any messages yet so I have no idea who has signed up, but I > would love to hear a frank discussion of currently available network > automation/configuration management (not monitoring) tools, both $$$ and > open source. > > I was interested by Brent's original NANOG posting, in which he states: > > ..."I'm not simply talking about [..] device configuration monitoring > systems [..] Instead I'm talking about systems that will start from a > description of how a network ought to be configured, then interact with > the various devices on that network to make it so..." > > What currently available tools actually do this? IMO the $$$ tools out > there today (or at least those I have seen referenced, ie the network > world article, rendition/Opsware, etc) all are pandering to the > sarbanes-oxley scare tactics because that's where the money is. If you > really look at them and peel away the marketing fluff and hand-waving, > they are all basically a configuration monitoring systems with a stamp on > the cd that says: your sarbanes-oxley problems solved here! Is their > goal even to solve network automation problems? I find it > ironically humorous that rendition renamed true control to "network > automation". Maybe they should have renamed it: Sarbanes-Oxley BandAid. > Not to pick on rendition, but the reality is there's money in checkbox > sarbanes-oxley solutions - and that's money in the right place (execs) and > a lot of it. So are real network management solutions getting left in the > dust? And what's worse at least IMO is that these guys all claim to > provide network management solutions, but don't actually provide value to > network administrators (or that isn't their main goal, its an > afterthought if its given any consideration at all), and because the money > is already spent, the network administrators don't get tools that might > actually solve a real problem. > > I hope this is just controversial enough to spur on some conversation, > because I would really like to hear everyone else's opinions and > experiences, and what else is out there that I haven't seen yet. > > Can anyone recommend open source tools for configuration > management/network automation, or is there interest in starting such a > project? > > > Thanks! > > > > From network-automation-owner@greatcircle.com Sat Apr 9 08:17:09 2005 X-Original-To: network-automation@greatcircle.com Received: from darksun.binsh.com (darksun.doomathon.com [64.232.216.23]) by mycroft.greatcircle.com (Postfix) with ESMTP id B610032C331 for ; Sat, 9 Apr 2005 08:17:08 -0700 (PDT) Received: from darksun.binsh.com (localhost [127.0.0.1]) by darksun.binsh.com (8.12.6/8.12.6) with ESMTP id j39FHp5C027737; Sat, 9 Apr 2005 08:17:51 -0700 (PDT) Received: from localhost (paxton@localhost) by darksun.binsh.com (8.12.6/8.12.6/Submit) with ESMTP id j39FHo2J027734; Sat, 9 Apr 2005 08:17:50 -0700 (PDT) X-Authentication-Warning: darksun.binsh.com: paxton owned process doing -bs Date: Sat, 9 Apr 2005 08:17:50 -0700 (PDT) From: Paxton To: Kirby Files Cc: Subject: Re: available network automation tools In-Reply-To: <42573546.7050503@gmail.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Archive-Number: 200504/10 X-Sequence-Number: 10 >> And the importance of having a queryable network model cannot be overstated. This has become the core of our OSS systems exactly. The vendor CM tools I've looked at all miss the boat in this regard. I don't want a bunch of one-hit-wonder tools, I want a foundation we can build from. I'm not sure any of these guys will get it right because they are going after what sells to the CIO, and the queryable network model is not sellable at that level (or so I'm told.) On Fri, 8 Apr 2005, Kirby Files wrote: > Paxton wrote on 04/08/2005 04:22 PM: > > I was interested by Brent's original NANOG posting, in which he states: > > > > ..."I'm not simply talking about [..] device configuration monitoring > > systems [..] Instead I'm talking about systems that will start from a > > description of how a network ought to be configured, then interact with > > the various devices on that network to make it so..." > > > > What currently available tools actually do this? > > None that I've seen. Before going the route of building our own > internal system for vendor-independent configuration generation and > management, I met with every CM vendor I could find, including a whole > bunch of dot-com busts. To name a few (the more promising ones): > > Orchestream > Dorado > Goldwire > Digital Fairways > Cplane > Network Mantra > Rendition > > Most of these focused on two things: config backup, and limited Cisco > service config templating. > > What we were looking for was a full network equipment and relationship > model, generic enough to model all types of routers, switches and > containers, along with a scriptable configuration generation engine to > interpret that model into device-dependent configuration deltas, and a > service activation manager to resolve dependencies (which services > required changes to which devices) and schedule config changes. > > Some of the tools above evolved to be support a little more than just > Cisco, and some of the config templating got a little more > sophisticated, but most have since died or been acquired, and none > meet the above goals. > > Unfortunately, 4 years later, we still wouldn't be able to replace our > homegrown configuration management system with any COTS system that > had comparable features. And the importance of having a queryable > network model cannot be overstated. This has become the core of our > OSS systems: driving data collection, automating NMS configuration and > performance measurement, and enabling much better inventory control > than we ever had before. > > Originally, I had hoped that the DMTF Common Information Model and > Directory Enabled Networking initiatives might yield some useful > products in this direction, where TMN failed to help. But these took > the same path of TMN of over-abstracting and over-specifying > particularly at the physical layer, making them unworkable for logical > network modeling. > > --kirby files > NMS Software Lead > Masergy Communications > From network-automation-owner@greatcircle.com Sat Apr 9 08:33:27 2005 X-Original-To: network-automation@greatcircle.com Received: from smtp-auth.no-ip.com (smtp-auth.no-ip.com [8.4.112.95]) by mycroft.greatcircle.com (Postfix) with SMTP id A77FC32C331 for ; Sat, 9 Apr 2005 08:33:26 -0700 (PDT) Received: (qmail 27817 invoked by uid 89); 9 Apr 2005 15:33:25 -0000 Received: from unknown (HELO localhost) (max.reid@saikonetworks.com@192.168.15.3) by mail1.no-ip.com with SMTP; 9 Apr 2005 15:33:25 -0000 Received: from wbar3.sea1-4-5-119-184.sea1.dsl-verizon.net (wbar3.sea1-4-5-119-184.sea1.dsl-verizon.net [4.5.119.184]) by webmail.no-ip.com (IMP) with HTTP for ; Sat, 9 Apr 2005 08:33:25 -0700 Message-ID: <1113060805.4257f5c567b57@webmail.no-ip.com> Date: Sat, 9 Apr 2005 08:33:25 -0700 From: max.reid@saikonetworks.com To: "Network.Security" Cc: network-automation@greatcircle.com Subject: Re: available network automation tools References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit User-Agent: Internet Messaging Program (IMP) 3.2.1 X-WebMail: No-IP.com X-Originating-IP: 4.5.119.184 X-Archive-Number: 200504/11 X-Sequence-Number: 11 Quoting "Network.Security" : > OK, I'll bite. > > We use Rendition (now Opsware) for config mgmt. of our network stuff and > the change detection / archival function has already saved my butt a > number of times. Our engineers gripe about the Big-Brother aspect, but > per the quasi-rant about SOX / CISP / PCI it's a fact of life now for > any SEC-filing corp at a minimum. > > I realize this is somewhat OT for this list, but SOX (in general) > shouldn't really matter to network admins (network meaning L1 to ~L4), > as SOX is all about altering the financial data. Hmm... I can understand the focus on the financial data, but doesn't it also cover stuff like email and all of IT under the umbrella of "Significant Security controls"? I know for a fact that storage vendors are drooling over the opportunity to sell SOX related warez and disks to PHBs everywhere. Regards, Max From network-automation-owner@greatcircle.com Sat Apr 9 17:45:40 2005 X-Original-To: network-automation@greatcircle.com Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.201]) by mycroft.greatcircle.com (Postfix) with ESMTP id 540D032C3FE for ; Sat, 9 Apr 2005 17:45:39 -0700 (PDT) Received: by rproxy.gmail.com with SMTP id r35so912108rna for ; Sat, 09 Apr 2005 17:45:38 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=BGQRKEVrK2cbdSFyYL4jPT+MRn+Ty//qME+w6MQeisf1yGTtr8fax0703Cq6WTFhVDLEo9S+FbrVjGYmZaGIr82Z2Sc2HPd/lSXtL8QcM9xcbM5Iywif58QPAfPvMReY235ZVJAtaIKqrDiGxzjlSKr1rECO95qLNnwKEJ18jPQ= Received: by 10.38.152.48 with SMTP id z48mr2830862rnd; Sat, 09 Apr 2005 17:45:38 -0700 (PDT) Received: by 10.38.151.34 with HTTP; Sat, 9 Apr 2005 17:45:38 -0700 (PDT) Message-ID: <18f60194050409174575ada600@mail.gmail.com> Date: Sat, 9 Apr 2005 17:45:38 -0700 From: Aaron Glenn Reply-To: Aaron Glenn To: Kirby Files Subject: Re: available network automation tools Cc: network-automation@greatcircle.com In-Reply-To: <42573546.7050503@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit References: <42573546.7050503@gmail.com> X-Archive-Number: 200504/12 X-Sequence-Number: 12 Kirby, If you would be so kind as to elaborate on a few of your points outlined below. On Apr 8, 2005 6:52 PM, Kirby Files wrote: > What we were looking for was a full network equipment and relationship > model, generic enough to model all types of routers, switches and > containers, along with a scriptable configuration generation engine to > interpret that model into device-dependent configuration deltas, and a > service activation manager to resolve dependencies (which services > required changes to which devices) and schedule config changes. Perhaps it's the cleaning fluid fumes in my apartment today, but I'm having trouble fully grasping what you're describing there. Can you take it down a notch to laymens terms? > Unfortunately, 4 years later, we still wouldn't be able to replace our > homegrown configuration management system with any COTS system that > had comparable features. And the importance of having a queryable > network model cannot be overstated. This has become the core of our > OSS systems: driving data collection, automating NMS configuration and > performance measurement, and enabling much better inventory control > than we ever had before. What is a "queryable network model" ? Thank you, aaron.glenn From network-automation-owner@greatcircle.com Sun Apr 10 08:03:45 2005 X-Original-To: network-automation@greatcircle.com Received: from mtxexch01.add0.masergy.com (unknown [64.47.104.21]) by mycroft.greatcircle.com (Postfix) with ESMTP id E026732C335 for ; Sun, 10 Apr 2005 08:03:43 -0700 (PDT) Received: from [64.47.15.101] ([64.47.15.101]) by mtxexch01.add0.masergy.com with Microsoft SMTPSVC(6.0.3790.211); Sun, 10 Apr 2005 10:03:01 -0500 Message-ID: <42593FE4.7030901@gmail.com> Date: Sun, 10 Apr 2005 11:01:56 -0400 From: Kirby Files User-Agent: Mozilla Thunderbird 1.0.1 (X11/20050309) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Aaron Glenn Cc: network-automation@greatcircle.com Subject: Re: available network automation tools References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> In-Reply-To: <18f60194050409174575ada600@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 10 Apr 2005 15:03:01.0616 (UTC) FILETIME=[63CA3F00:01C53DDE] X-Archive-Number: 200504/13 X-Sequence-Number: 13 Aaron Glenn wrote on 04/09/2005 08:45 PM: > Kirby, > > If you would be so kind as to elaborate on a few of your points outlined below. > > On Apr 8, 2005 6:52 PM, Kirby Files wrote: > >>What we were looking for was a full network equipment and relationship >>model, generic enough to model all types of routers, switches and >>containers, along with a scriptable configuration generation engine to >>interpret that model into device-dependent configuration deltas, and a >>service activation manager to resolve dependencies (which services >>required changes to which devices) and schedule config changes. > > > Perhaps it's the cleaning fluid fumes in my apartment today, but I'm > having trouble fully grasping what you're describing there. Can you > take it down a notch to laymens terms? For us, a complete configuration management system includes: * a vendor-independent model of the physical and logical network (objects and fields describing configuration of devices, cards, interfaces, etc; and relationships between these objects) * An interface for defining this model in network engineering terms * definition of services in terms of component devices and logical path * scripts or code defining how to build configs for specific vendors based upon the model, connections, and applied services * service activation manager that can manage many simultaneous threads of network communication for deploying these configs in a transactional process * configuration backups, history, diffs, equipment hardware management, etc. * network auditing to ensure database accuracy * a database schema or API to query all fields, relationships, events, etc. This combination allows for the shifting of all configuration responsibilities from users to the CM system (now, users mostly login to core equipment only for troubleshooting), speeding up deployment, reducing errors, and improving reliability. It also serves to provide a true functional definition for all services frequently lacking in Product Mangement docs; if the product can't be reduced to a well-defined body of script or code, it's not baked enough to be called "standard". > What is a "queryable network model" ? Well, as described above, the network model is a database that is capable of describing what the equipment is, how it is configured, and how devices are connected to each other. It includes the network topologies and hierarchy, logical connections, IP subnets, routing policies, physical components, etc. For us, the key has been to keeps the schema for the model defined not in the management software or static database tables, but a NetEng modifiable syntax that can be quickly changed to add new concepts, hardware, and services. In this context, queryable means just that: all interesting features of the network should be discoverable by external users and systems, rather than being hardcoded in software. Whether this is implemented in SQL (not always recommended, especially since SQL is terrible at tree representations), a well-defined API, query syntax, or message format is less important. Thanks, --kirby From network-automation-owner@greatcircle.com Sun Apr 10 21:49:16 2005 X-Original-To: network-automation@greatcircle.com Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.201]) by mycroft.greatcircle.com (Postfix) with ESMTP id 096A332C446 for ; Sun, 10 Apr 2005 21:49:11 -0700 (PDT) Received: by rproxy.gmail.com with SMTP id r35so1103743rna for ; Sun, 10 Apr 2005 21:49:10 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=MSMF9Z42NayzbqKkEeoCcE+V5jjyGNGD7OYAcSDUVCLkA89HAAg6eVrunjFMklrQNz0ODUu9KBPH4NJAV7t6/ouOzEf0s7mcPX96VG+yMXIdelYtdCA1asPNFaQXn5QfgIBrgaphVT8acHOPy7J0IaGNtmoqo5cuDIKb1MZaAds= Received: by 10.38.90.72 with SMTP id n72mr4896607rnb; Sun, 10 Apr 2005 21:49:10 -0700 (PDT) Received: by 10.39.1.43 with HTTP; Sun, 10 Apr 2005 21:49:10 -0700 (PDT) Message-ID: <7654d9d05041021497c88c860@mail.gmail.com> Date: Mon, 11 Apr 2005 14:49:10 +1000 From: Andrew Fort Reply-To: Andrew Fort To: network-automation@greatcircle.com Subject: Re: available network automation tools In-Reply-To: <42593FE4.7030901@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> <42593FE4.7030901@gmail.com> X-Archive-Number: 200504/14 X-Sequence-Number: 14 On Apr 11, 2005 1:01 AM, Kirby Files wrote: > For us, a complete configuration management system includes: > * a vendor-independent model of the physical and logical network > (objects and fields describing configuration of devices, cards, > interfaces, etc; and relationships between these objects) > * An interface for defining this model in network engineering terms > * definition of services in terms of component devices and logical path > * scripts or code defining how to build configs for specific vendors > based upon the model, connections, and applied services > * service activation manager that can manage many simultaneous > threads of network communication for deploying these configs in a > transactional process > * configuration backups, history, diffs, equipment hardware > management, etc. > * network auditing to ensure database accuracy > * a database schema or API to query all fields, relationships, > events, etc. > > This combination allows for the shifting of all configuration > responsibilities from users to the CM system (now, users mostly login > to core equipment only for troubleshooting), speeding up deployment, > reducing errors, and improving reliability. It also serves to provide > a true functional definition for all services frequently lacking in > Product Mangement docs; if the product can't be reduced to a > well-defined body of script or code, it's not baked enough to be > called "standard". All spot on. I particularly like the product definition one, that has played an important role here already - and we've only gotten to the stage of importing network configuration into the model and highlighting further study cases... The key steps and comments in building and implementing such a system, which many are faced with doing (hence: this is a great focus for an open source project I think :-)), are, I see (and this is the process we're going through now where I work): - Model the network. In one projection of the idea, at the simplest level you have objects and relationships (which are objects themselves). Both of these types have attributes. It's important to be able to map any network type that you'll need to configure, so you probably need to store VLAN/FrDLCI information, for example. How to map that? Perhaps some recursion is useful (a network segment object (e.g. DLCI, VLAN) is "a member of" or "attached to" a physical link object. IP interfaces are just mapped to a logical or physical parent, and so on. The key is obviously to get a model which can represent your entire network completely enough that you don't need to work around it. The basic approach above will work, how far you abstract it is network dependant. - Understand your deployment characteristics. You may or may not have out-of-band management (in some and not all locations). If not, how do you avoid cases like disabling the IGP router so that you won't break the telnet session as the config is applied? How then, do you perform changes to vital (e.g. management) interface configuration? How do you avoid such problems? "disable certain commands" immediately comes to mind, but how can you allow certain configuration to occur whilst disabling potentially important commands. This part is inevitably somewhat network specific, but you may like to use approaches to have an interface attribute such as "management". Configuration for these interfaces is then manual. However, it may be impossible to know of all of these cases ahead of time without an exhaustive search (possibly related to strange loops/Godel's incompleteness theorem?). Then you'll need some way of applying the config to the devices. A device agent would be best (since we can treat the change to happen atomically, at some particular pre-scheduled time) (cisco have such an agent, but the interface is closed), at worst scripted-VTY access. The latter of course has the problems spoken about above. Scripted-console access may be a real boon, but is largely unsuitable for many devices (CPE, small sites depending on your management "will"), so we'll have to consider scripted VTY or similar - SNMP configuration MIBs are not an option here (no non-v3 write (policy)), but may be for you.. - Finally, enforce the model as being more important than the element. The network element just provides reality for your network model. Process as well as policy is needed (I encourage TAC+ command lockouts), Other notes I have about the goals of the system are: - Configuration applied must have the same action no matter what the initial state of the target element. This generally translates to "bring all pre-requisite configuration along with the change". This is important so that you can apply partial configurations to devices in whatever state -- many of the "big boys" who do network configuration management presently load to startup and reload, and their networks handle this type of management. For many of us out there, that's not possible. So the model must support storing all necessary configuration (or at least, identifying what standard templates need to be placed in what order to build the "base config" of the device) so that both partial and complete configurations can be formed. How do I know what is already configured on the device? Assume nothing? Probably. This depends on if you want to completely trust your database, or whether you have some trust in the existing configuration. The more I work in this area, the less I can trust the network configuration. - Allow the network operators to easily identify the change (via some ID) that caused the most recent (or couple of most recent) change to the element configuration. This tends to be important in helping the NOC identify the triggers for recent change to the configuration. You can alert people of this in other ways, e.g. via the configuration manager itself. Auto-generating "banner exec" works well on cisco devices, for example, as a sign that "the change was applied" (whether it was successful or not, is somewhat different). Once the above has been nutted out, the multi-vendor hurdles are mostly irrelevant, and all related to the "deployment characteristics" situation above. -andrew From network-automation-owner@greatcircle.com Mon Apr 11 07:55:28 2005 X-Original-To: network-automation@greatcircle.com Received: from perdition.linnaean.org (dsl092-073-217.bos1.dsl.speakeasy.net [66.92.73.217]) by mycroft.greatcircle.com (Postfix) with ESMTP id 101D932C452 for ; Sun, 10 Apr 2005 23:24:35 -0700 (PDT) Received: by perdition.linnaean.org (Postfix, from userid 31013) id 9614119343; Mon, 11 Apr 2005 02:24:30 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16986.6174.403374.979347@perdition.linnaean.org> Date: Mon, 11 Apr 2005 02:24:30 -0400 To: Andrew Fort Cc: network-automation@greatcircle.com Subject: Re: available network automation tools In-Reply-To: <7654d9d05041021497c88c860@mail.gmail.com> References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> <42593FE4.7030901@gmail.com> <7654d9d05041021497c88c860@mail.gmail.com> X-Mailer: VM 7.19 under Emacs 20.7.1 Reply-To: Daniel Hagerty From: Daniel Hagerty X-Archive-Number: 200504/15 X-Sequence-Number: 15 > Configuration for these interfaces is then manual. However, it may be > impossible to know of all of these cases ahead of time without an > exhaustive search (possibly related to strange loops/Godel's > incompleteness theorem?). "It depends". On one extreme, you limit the language you can speak, preventing the ugly cases, consequently limiting the capability of your model. On the other is no restriction, with a resultingly more expressive model, but with halting trouble if the user doesn't use it well. Personally, I'm for being allowed to shoot myself in the head if that's what I said to do, but getting to a model that allows me to both shoot myself in the head and have the computer warn me that "by the way, you might want to know that saying 'twiddle the flubbypux into state 29' may mean 'shoot yourself and twenty friends in the head'" is a little bit of work. Witness my example from a few days ago on infrastructures: the addresses within a subnet are numbers. I don't mean that in the obvious sense (being 32 bit numbers), I mean that in a more subtle one. If you allow the user of your configuration system to operate on these numbers in arbitrary ways and define functions on them (which you pretty much have to do in order to give them what they need), you're giving them the very tools they need to create halting problems. In the case of network engineering, your semantic and reflection problems with reasoning with loops is relatively easy. Consider the case of destroying the only path to talk to a computer (not terribly different from a router in this simple example). Assuming a linux machine, and we were sitting at some other host in our network, typing something like the following is an instance of the statement class we are concerned with: $ ssh host-thats-gonna-die $ sudo ifconfig eth0 0 where once authenticate2 completes, the deed is done. What we want to be able to do is equate the above statement with the notion of destroying our ability to reach hosts-thats-gonna-die from wherever we started from (and if our network has reasonable transitive properties, no host can reach it). This is "simple", for an annoying value of simple. It mostly comes down to being able to compute the meaning of each statement in a reasonably large size world (one thats as large as your network). In particular, you need to be able to reason with how "sudo ifconfig eth0 0" transmitted to our victim host is going to side effect the network configuration of the victim such that it would be unreachable to the broader world. Somehow differentiate this condition from other cases like "sudo ifconfig eth0 192.168.1.1 netmask 0xffffff00" where that isn't a reasonable value for your broader network and "sudo ifconfig eth0 192.168.2.1 netmask 0xffffff00" where that just so happens to be the same as what's configured now. May of the issues you speak of are creations of the semantic issues of configuration management. Here's the basic ordering that the math enforces: 1. Do the following, in parallel. They do not readily seperate. a. Figure out the semantic problems. b. Figure out the knowledge problems. 2. Arrive at something that works for your domain. Try to apply it to another domain. Watch it break. Go back to step 1. So, "model the network" is really step 1 in disguise. Understanding of your deployment characteristics is once again step 1. The question of how to avoid disabling IGP when you're using telnet that depends on routing working to perform the configuration changes is a knowledge and semantics question. Some amount of the questions of "how to produce delta statements", both at some macro level of the entire network, and the micro level of a given device are also semantic in nature. There are lot of other issues involved, but questions in the form of "how does it behave" and "how do I tell it...?" should set of bells. This is not to say that you are in an endless sea of "figure out step one first or you're doomed". Quite the opposite; take the problem apart in whatever way you find works, and there are plenty of parts out there. Hopefully some of my own examples help. I'm not sure how tall we'll have to climb to really produce satisfactory reasoning with the problem of "this" yet. The IGP example, the destroy the ip address of the interface, and many other complex "godel" looking problems in configuration come down to the presence of an implied "this" concept. For simple things like "this router", it's can be easy enough to express. Other things are probably less so. From network-automation-owner@greatcircle.com Mon Apr 11 08:18:51 2005 X-Original-To: network-automation@greatcircle.com Received: from darksun.binsh.com (darksun.doomathon.com [64.232.216.23]) by mycroft.greatcircle.com (Postfix) with ESMTP id 630F332C36F for ; Mon, 11 Apr 2005 08:18:50 -0700 (PDT) Received: from darksun.binsh.com (localhost [127.0.0.1]) by darksun.binsh.com (8.12.6/8.12.6) with ESMTP id j3BFJZ5C029420 for ; Mon, 11 Apr 2005 08:19:35 -0700 (PDT) Received: from localhost (paxton@localhost) by darksun.binsh.com (8.12.6/8.12.6/Submit) with ESMTP id j3BFJUas029417 for ; Mon, 11 Apr 2005 08:19:35 -0700 (PDT) X-Authentication-Warning: darksun.binsh.com: paxton owned process doing -bs Date: Mon, 11 Apr 2005 08:19:30 -0700 (PDT) From: Paxton To: Subject: Re: available network automation tools In-Reply-To: <7654d9d05041021497c88c860@mail.gmail.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Archive-Number: 200504/16 X-Sequence-Number: 16 on network modeling, have you all come up with your own model or used something existing? Or taken something existing and modified it? I have looked at CIM and after scratching my head for a few days finally realized that I think of the network in terms of associations, and CIM is laid out by generalizations (inheritance.) The associations are defined, but are sort of hard to find because of the generalization perspective. I don't want to get into a semantics discussion of what's right, I'm more interested in being able to hand a database to a network engineer and say: go for it. IMO a db schema laid out with CIM is too confusing because it isn't laid out according to associations, which is (IMO) how network engineers (vs IT modeling guys) see things. Anyway, I have done this before and we sort of took some clues from what was available at the time (SNMP and some vendor tools.) It wasn't exactly right, but I learned a lot from the experience. We put together the database for one specific tool, then realized afterwards how much else it could be used for. I believe the core piece missing is the network model/database, consider all the other parts (how to update devices, monitoring configuration changes, etc) as component services that the database enables. Not all the component parts have to be written for the database to have value, and not everyone will want to use the same component parts. Are you all interested in starting an open source project around this? On Mon, 11 Apr 2005, Andrew Fort wrote: > On Apr 11, 2005 1:01 AM, Kirby Files wrote: > > > For us, a complete configuration management system includes: > > * a vendor-independent model of the physical and logical network > > (objects and fields describing configuration of devices, cards, > > interfaces, etc; and relationships between these objects) > > * An interface for defining this model in network engineering terms > > * definition of services in terms of component devices and logical path > > * scripts or code defining how to build configs for specific vendors > > based upon the model, connections, and applied services > > * service activation manager that can manage many simultaneous > > threads of network communication for deploying these configs in a > > transactional process > > * configuration backups, history, diffs, equipment hardware > > management, etc. > > * network auditing to ensure database accuracy > > * a database schema or API to query all fields, relationships, > > events, etc. > > > > This combination allows for the shifting of all configuration > > responsibilities from users to the CM system (now, users mostly login > > to core equipment only for troubleshooting), speeding up deployment, > > reducing errors, and improving reliability. It also serves to provide > > a true functional definition for all services frequently lacking in > > Product Mangement docs; if the product can't be reduced to a > > well-defined body of script or code, it's not baked enough to be > > called "standard". > > All spot on. I particularly like the product definition one, that has > played an important role here already - and we've only gotten to the > stage of importing network configuration into the model and > highlighting further study cases... > > The key steps and comments in building and implementing such a system, > which many are faced with doing (hence: this is a great focus for an > open source project I think :-)), are, I see (and this is the process > we're going through now where I work): > > - Model the network. In one projection of the idea, at the simplest > level you have objects and relationships (which are objects > themselves). Both of these types have attributes. It's important to > be able to map any network type that you'll need to configure, so you > probably need to store VLAN/FrDLCI information, for example. How to > map that? Perhaps some recursion is useful (a network segment object > (e.g. DLCI, VLAN) is "a member of" or "attached to" a physical link > object. IP interfaces are just mapped to a logical or physical > parent, and so on. > > The key is obviously to get a model which can represent your entire > network completely enough that you don't need to work around it. The > basic approach above will work, how far you abstract it is network > dependant. > > - Understand your deployment characteristics. You may or may not > have out-of-band management (in some and not all locations). If not, > how do you avoid cases like disabling the IGP router so that you won't > break the telnet session as the config is applied? How then, do you > perform changes to vital (e.g. management) interface configuration? > > How do you avoid such problems? "disable certain commands" > immediately comes to mind, but how can you allow certain configuration > to occur whilst disabling potentially important commands. This part > is inevitably somewhat network specific, but you may like to use > approaches to have an interface attribute such as "management". > Configuration for these interfaces is then manual. However, it may be > impossible to know of all of these cases ahead of time without an > exhaustive search (possibly related to strange loops/Godel's > incompleteness theorem?). > > Then you'll need some way of applying the config to the devices. A > device agent would be best (since we can treat the change to happen > atomically, at some particular pre-scheduled time) (cisco have such an > agent, but the interface is closed), at worst scripted-VTY access. > The latter of course has the problems spoken about above. > Scripted-console access may be a real boon, but is largely unsuitable > for many devices (CPE, small sites depending on your management > "will"), so we'll have to consider scripted VTY or similar - SNMP > configuration MIBs are not an option here (no non-v3 write (policy)), > but may be for you.. > > - Finally, enforce the model as being more important than the > element. The network element just provides reality for your network > model. Process as well as policy is needed (I encourage TAC+ command > lockouts), > > Other notes I have about the goals of the system are: > > - Configuration applied must have the same action no matter what the > initial state of the target element. This generally translates to > "bring all pre-requisite configuration along with the change". > > This is important so that you can apply partial configurations to > devices in whatever state -- many of the "big boys" who do network > configuration management presently load to startup and reload, and > their networks handle this type of management. For many of us out > there, that's not possible. > > So the model must support storing all necessary configuration (or at > least, identifying what standard templates need to be placed in what > order to build the "base config" of the device) so that both partial > and complete configurations can be formed. > > How do I know what is already configured on the device? Assume > nothing? Probably. This depends on if you want to completely trust > your database, or whether you have some trust in the existing > configuration. The more I work in this area, the less I can trust the > network configuration. > > - Allow the network operators to easily identify the change (via some > ID) that caused the most recent (or couple of most recent) change to > the element configuration. This tends to be important in helping the > NOC identify the triggers for recent change to the configuration. You > can alert people of this in other ways, e.g. via the configuration > manager itself. > > Auto-generating "banner exec" works well on cisco devices, for > example, as a sign that "the change was applied" (whether it was > successful or not, is somewhat different). > > Once the above has been nutted out, the multi-vendor hurdles are > mostly irrelevant, and all related to the "deployment characteristics" > situation above. > > -andrew > From network-automation-owner@greatcircle.com Mon Apr 11 08:44:05 2005 X-Original-To: network-automation@greatcircle.com Received: from [66.92.48.19] (localhost [127.0.0.1]) by mycroft.greatcircle.com (Postfix) with ESMTP id 885C032C16B for ; Mon, 11 Apr 2005 08:44:04 -0700 (PDT) Mime-Version: 1.0 Message-Id: In-Reply-To: References: Date: Mon, 11 Apr 2005 08:44:01 -0700 To: From: Brent Chapman Subject: Re: available network automation tools Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Archive-Number: 200504/17 X-Sequence-Number: 17 At 3:56 PM -0500 4/8/05, Network.Security wrote: >Now, CISP / PCI on the other hand, that's the project funding behemoth >you've all been waiting for. If you need money, say it's for PCI and >poof-like-magic, here's the cash to make it happen. PCI has some fairly >strict requirements that are defined to the network level regarding open >ports, encryption schemes, use of clear-text, etc. Tools like Opsware >can help enforce or at least notify on those data points. What are CISP and PCI? >Opsware and Voyence (we demo-ed them) both do some configuration >templating so that if you are building out CPE, you can have it make >your router configs or whatever, but for already installed networks, the >templating is not valuable. I'm all about the configuration policy >piece, I want to know how many of my devices don't have enough ntp >servers configured, that sort of thing or down the road to make sure my >QoS policies are consistent across the board. But that's all to get me >to a point in time where everything is "right", it doesn't help me >deploy new services all that differently than my perl scripts did >before. If all you're using config generation for is CPE (branch office and employee home devices, for example), rather than your core systems, then I don't think you're ever going to reap the benefits that I'm contemplating. The trick, of course, is moving an existing network from manually-configured, manually-maintained (and grossly inconsistent) configurations to automated configurations without disrupting service. It's hard, but I think it can be done, and is worthwhile to do so. >What I think you are talking about is an application aware network >provisioning system. Something that is aware of all possible >topological paths between endpoints and is smart enough to know how to >configure all the hops / connections in between the two to make >something happen. Like a DOS mitigation system or punching holes >through a series of firewalls or configuring multi-hop VPN tunnels. No, I was talking about something more general, although what you're describing are examples of things ("applications", in a sense) that you could build on top of what I'm talking about. >BTW, Opsware and others would love to see this list if it grows and >utilize it as a tool for soliciting industry-wide customer feedback. I >would suggest consideration of that either for it or against (I don't >really care) and noting it in the policy. Vendors and developers are welcome to participate or lurk here, although blatant and content-free self-promotion will be met with ridicule from the audience (their potential customers). If you've got any technical or product management contacts at various relevant vendors, please let them know about the list: http://www.greatcircle.com/network-automation/ -Brent -- Brent Chapman -- Great Circle Associates, Inc. Specializing in network infrastructure for Silicon Valley since 1989 For info about us and our services, please see http://www.greatcircle.com/ Network Automation blog: http://www.greatcircle.com/blog/network_automation From network-automation-owner@greatcircle.com Mon Apr 11 08:46:10 2005 X-Original-To: network-automation@greatcircle.com Received: from perdition.linnaean.org (dsl092-073-217.bos1.dsl.speakeasy.net [66.92.73.217]) by mycroft.greatcircle.com (Postfix) with ESMTP id EAFE032C320 for ; Mon, 11 Apr 2005 08:46:08 -0700 (PDT) Received: by perdition.linnaean.org (Postfix, from userid 31013) id 7BC3A19342; Mon, 11 Apr 2005 11:46:08 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16986.39872.398150.16825@perdition.linnaean.org> Date: Mon, 11 Apr 2005 11:46:08 -0400 To: Andrew Fort Cc: network-automation@greatcircle.com Subject: Re: available network automation tools In-Reply-To: <7654d9d05041021497c88c860@mail.gmail.com> References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> <42593FE4.7030901@gmail.com> <7654d9d05041021497c88c860@mail.gmail.com> X-Mailer: VM 7.19 under Emacs 20.7.1 Reply-To: Daniel Hagerty From: Daniel Hagerty X-Archive-Number: 200504/18 X-Sequence-Number: 18 [ Resending after majordomo decided it didn't like me. Forgive me if you've seen this before. ] > Configuration for these interfaces is then manual. However, it may be > impossible to know of all of these cases ahead of time without an > exhaustive search (possibly related to strange loops/Godel's > incompleteness theorem?). "It depends". On one extreme, you limit the language you can speak, preventing the ugly cases, consequently limiting the capability of your model. On the other is no restriction, with a resultingly more expressive model, but with halting trouble if the user doesn't use it well. Personally, I'm for being allowed to shoot myself in the head if that's what I said to do, but getting to a model that allows me to both shoot myself in the head and have the computer warn me that "by the way, you might want to know that saying 'twiddle the flubbypux into state 29' may mean 'shoot yourself and twenty friends in the head'" is a little bit of work. Witness my example from a few days ago on infrastructures: the addresses within a subnet are numbers. I don't mean that in the obvious sense (being 32 bit numbers), I mean that in a more subtle one. If you allow the user of your configuration system to operate on these numbers in arbitrary ways and define functions on them (which you pretty much have to do in order to give them what they need), you're giving them the very tools they need to create halting problems. In the case of network engineering, your semantic and reflection problems with reasoning with loops is relatively easy. Consider the case of destroying the only path to talk to a computer (not terribly different from a router in this simple example). Assuming a linux machine, and we were sitting at some other host in our network, typing something like the following is an instance of the statement class we are concerned with: $ ssh host-thats-gonna-die $ sudo ifconfig eth0 0 where once authenticate2 completes, the deed is done. What we want to be able to do is equate the above statement with the notion of destroying our ability to reach hosts-thats-gonna-die from wherever we started from (and if our network has reasonable transitive properties, no host can reach it). This is "simple", for an annoying value of simple. It mostly comes down to being able to compute the meaning of each statement in a reasonably large size world (one thats as large as your network). In particular, you need to be able to reason with how "sudo ifconfig eth0 0" transmitted to our victim host is going to side effect the network configuration of the victim such that it would be unreachable to the broader world. Somehow differentiate this condition from other cases like "sudo ifconfig eth0 192.168.1.1 netmask 0xffffff00" where that isn't a reasonable value for your broader network and "sudo ifconfig eth0 192.168.2.1 netmask 0xffffff00" where that just so happens to be the same as what's configured now. May of the issues you speak of are creations of the semantic issues of configuration management. Here's the basic ordering that the math enforces: 1. Do the following, in parallel. They do not readily seperate. a. Figure out the semantic problems. b. Figure out the knowledge problems. 2. Arrive at something that works for your domain. Try to apply it to another domain. Watch it break. Go back to step 1. So, "model the network" is really step 1 in disguise. Understanding of your deployment characteristics is once again step 1. The question of how to avoid disabling IGP when you're using telnet that depends on routing working to perform the configuration changes is a knowledge and semantics question. Some amount of the questions of "how to produce delta statements", both at some macro level of the entire network, and the micro level of a given device are also semantic in nature. There are lot of other issues involved, but questions in the form of "how does it behave" and "how do I tell it...?" should set of bells. This is not to say that you are in an endless sea of "figure out step one first or you're doomed". Quite the opposite; take the problem apart in whatever way you find works, and there are plenty of parts out there. Hopefully some of my own examples help. I'm not sure how tall we'll have to climb to really produce satisfactory reasoning with the problem of "this" yet. The IGP example, the destroy the ip address of the interface, and many other complex "godel" looking problems in configuration come down to the presence of an implied "this" concept. For simple things like "this router", it's can be easy enough to express. Other things are probably less so. From network-automation-owner@greatcircle.com Mon Apr 11 09:15:38 2005 X-Original-To: network-automation@greatcircle.com Received: from iad2smtpout02.target.com (smtpout03.target.com [161.225.129.10]) by mycroft.greatcircle.com (Postfix) with ESMTP id 125EE32C36D for ; Mon, 11 Apr 2005 09:15:37 -0700 (PDT) Received: from emailhub03.target.com ([10.114.73.110]) by iad2smtpout02.target.com with Microsoft SMTPSVC(6.0.3790.211); Mon, 11 Apr 2005 11:15:36 -0500 Received: from EMAILSTORE13.target.com ([10.114.81.32]) by emailhub03.target.com with Microsoft SMTPSVC(6.0.3790.211); Mon, 11 Apr 2005 11:15:35 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: Re: available network automation tools Date: Mon, 11 Apr 2005 11:15:34 -0500 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: available network automation tools Thread-Index: AcU+rU+zA/Us/xdzSSqbTA5LF4UxqQAAbHKQ From: "Network.Security" To: X-OriginalArrivalTime: 11 Apr 2005 16:15:35.0927 (UTC) FILETIME=[B1932070:01C53EB1] X-Archive-Number: 200504/19 X-Sequence-Number: 19 >What are CISP and PCI? CISP =3D Cardholder Information Security Program (Visa specific) and PCI = =3D Payment Card Industry Data Security Standard. These govern the security of credit cardholder data, the transmission of that data, the security practices and polices around it, etc. This was an effort started by Visa and later adopted by the major CC companies into a consolidated security audit. Companies that process CC data are evaluated and later penalized if they do not meet criteria within a given timeframe. The penalties can be navigated around if you present a solid plan, that sort of thing. Further, it outlines minimum financial penalties for security breaches and release of CC data (aka hacked accounts). For retailers and other entities that depend on CC data, this is huge. For an industry audit, it's fairly detailed in that it talks about specific TCP ports, encryption schemes, etc. Info at: www.visa.com/cisp =20 My comments re: SOX were not meant to imply that it doesn't matter, more that there are multiple sources of funding for compliance related projects, some may be more applicable than others. - Scott scott.altman@target.com From network-automation-owner@greatcircle.com Mon Apr 11 16:11:51 2005 X-Original-To: Network-Automation@GreatCircle.COM Received: from [66.92.48.19] (localhost [127.0.0.1]) by mycroft.greatcircle.com (Postfix) with ESMTP id 021DE32C4D2 for ; Mon, 11 Apr 2005 16:11:50 -0700 (PDT) Mime-Version: 1.0 Message-Id: Date: Mon, 11 Apr 2005 16:11:46 -0700 To: Network-Automation@GreatCircle.COM From: Brent Chapman Subject: NetML? Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Archive-Number: 200504/20 X-Sequence-Number: 20 At 8:05 AM -0700 4/9/05, max.reid@saikonetworks.com wrote: >There certainly isn't a cfengine type application for multivendor >network devices, but with cisco Incorporating Tcl shells and >precompilers in IOS, F5 using a Tcl shell to replace their >proprietary scripting language, There just needs to be an Open >Reference model or XML schema to describe device functions... I >don't think we're there yet. Has anybody take a look at NetML? http://giga.dia.uniroma3.it/~ivan/NetML/intro.html NetML is a network description/configuration markup language. It is based on the eXtensible Markup Language (XML). Its main aims are to describe a network at Autonomous System level or at a lower level (i.e. ISO-OSI level 2) and to be able to configure Cisco, Juniper or Zebra routers. Using the eXtensible Stylesheet Language Transformations, it is possible to produce the configuration files for the various types of routers. The tools produced in this project allow the writing of the NetML file (using a browser) and the production of the configuration files for the various constructors starting from the NetML file. See also: http://www.ripe.net/ripe/meetings/ripe-47/eof.html#netml http://www.netkit.org/ -Brent -- Brent Chapman -- Great Circle Associates, Inc. Specializing in network infrastructure for Silicon Valley since 1989 For info about us and our services, please see http://www.greatcircle.com/ Network Automation blog: http://www.greatcircle.com/blog/network_automation From network-automation-owner@greatcircle.com Mon Apr 11 16:15:14 2005 X-Original-To: network-automation@greatcircle.com Received: from [66.92.48.19] (localhost [127.0.0.1]) by mycroft.greatcircle.com (Postfix) with ESMTP id 1673432C4D8; Mon, 11 Apr 2005 16:15:13 -0700 (PDT) Mime-Version: 1.0 Message-Id: In-Reply-To: References: Date: Mon, 11 Apr 2005 16:15:10 -0700 To: Paxton , Kirby Files From: Brent Chapman Subject: Re: available network automation tools Cc: Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Archive-Number: 200504/21 X-Sequence-Number: 21 At 8:17 AM -0700 4/9/05, Paxton wrote: > >> And the importance of having a queryable >network model cannot be overstated. This has become the core of our >OSS systems > >exactly. The vendor CM tools I've looked at all miss the boat in this >regard. I don't want a bunch of one-hit-wonder tools, I want >a foundation we can build from. Hear, hear! This is the sort of thing I've been thinking about, too. >I'm not sure any of these guys will get >it right because they are going after what sells to the CIO, and the >queryable network model is not sellable at that level (or so I'm told.) Perhaps not directly sellable, but the features and benefits of tools built atop such a model (modularity, expandability, consistency, predictability, efficiency, etc.) ought to be. -Brent -- Brent Chapman -- Great Circle Associates, Inc. Specializing in network infrastructure for Silicon Valley since 1989 For info about us and our services, please see http://www.greatcircle.com/ Network Automation blog: http://www.greatcircle.com/blog/network_automation From network-automation-owner@greatcircle.com Mon Apr 11 16:30:23 2005 X-Original-To: network-automation@greatcircle.com Received: from [66.92.48.19] (localhost [127.0.0.1]) by mycroft.greatcircle.com (Postfix) with ESMTP id 7DE4A32C4DC; Mon, 11 Apr 2005 16:30:21 -0700 (PDT) Mime-Version: 1.0 Message-Id: In-Reply-To: <42593FE4.7030901@gmail.com> References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> <42593FE4.7030901@gmail.com> Date: Mon, 11 Apr 2005 16:30:18 -0700 To: Kirby Files , Aaron Glenn From: Brent Chapman Subject: Re: available network automation tools Cc: network-automation@greatcircle.com Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Archive-Number: 200504/22 X-Sequence-Number: 22 At 11:01 AM -0400 4/10/05, Kirby Files wrote: >Aaron Glenn wrote on 04/09/2005 08:45 PM: > > Perhaps it's the cleaning fluid fumes in my apartment today, but I'm >> having trouble fully grasping what you're describing there. Can you >> take it down a notch to laymens terms? > >For us, a complete configuration management system includes: > * a vendor-independent model of the physical and logical network >(objects and fields describing configuration of devices, cards, >interfaces, etc; and relationships between these objects) In other words, we want a model that represents information like: - Device D1 contains cards C1, C2, and C3 - Card C1 on Device X has ports P1, P2, P3, and P4 of type Y - Port P1 on Card C1 on Device D1 is a 100 Mb/s Ethernet port - Port P1 on Card C1 on Device D1 is connected to network N1 - Network N1 is 192.168.1.0/24 - Port P1 on Card C1 on Device D1 is 192.168.1.1/24 And so forth, in as much detail as you want, for all the various parts of your network. > * An interface for defining this model in network engineering terms In other words, an interface that lets you do things like - Add Device - Add Card to Device - Add Port to Card - Add Network - Describe connection from Port to Network etc. > * definition of services in terms of component devices and logical path I think I know what Kirby is talking about here, but I can't think of a simple example. > * scripts or code defining how to build configs for specific vendors >based upon the model, connections, and applied services In other words, code that knows how to generate a config file for Device D1 if that device is a Cisco, based on data drawn from the model. The code has to be able to extract information from the model such as: - What make/model of device is D1? - What interfaces exist on D1? - What ports are in use on those interfaces? - What networks are those ports connected to? - What are the characteristics/parameters of those connections? > * service activation manager that can manage many simultaneous >threads of network communication for deploying these configs in a >transactional process At this point, we move beyond the model to the actual ability to deploy configurations derived from the model. In other words, I conceptually separate the problem into two parts: 1) What should the configs be for each device? 2) How do I get those configs onto those devices? > * configuration backups, history, diffs, equipment hardware >management, etc. In other words, the ability to answer questions like: - What was the configuration of device D1 yesterday, before we changed things and started having problems? - How many unused card slots are there in device D1? - How many unused ports are there on Card C1 in device D1? > * network auditing to ensure database accuracy In other words, "is my model of the network accurate?" > * a database schema or API to query all fields, relationships, >events, etc. So that you can write the code to generate the configs for the devices, as well as generate configs for related tools like SNMP monitors (Cricket, etc.), system status monitors (Nagios, etc.) >This combination allows for the shifting of all configuration >responsibilities from users to the CM system (now, users mostly login >to core equipment only for troubleshooting), speeding up deployment, >reducing errors, and improving reliability. It also serves to provide >a true functional definition for all services frequently lacking in >Product Mangement docs; if the product can't be reduced to a >well-defined body of script or code, it's not baked enough to be >called "standard". Yup. > > What is a "queryable network model" ? > >Well, as described above, the network model is a database that is >capable of describing what the equipment is, how it is configured, and >how devices are connected to each other. It includes the network >topologies and hierarchy, logical connections, IP subnets, routing >policies, physical components, etc. > >For us, the key has been to keeps the schema for the model defined not >in the management software or static database tables, but a NetEng >modifiable syntax that can be quickly changed to add new concepts, >hardware, and services. Yes, this is important; it's got to be extendable, because you aren't going to think of everything up front. -Brent -- Brent Chapman -- Great Circle Associates, Inc. Specializing in network infrastructure for Silicon Valley since 1989 For info about us and our services, please see http://www.greatcircle.com/ Network Automation blog: http://www.greatcircle.com/blog/network_automation From network-automation-owner@greatcircle.com Mon Apr 11 16:35:29 2005 X-Original-To: network-automation@greatcircle.com Received: from [66.92.48.19] (localhost [127.0.0.1]) by mycroft.greatcircle.com (Postfix) with ESMTP id 0C2CA32C4D8; Mon, 11 Apr 2005 16:35:28 -0700 (PDT) Mime-Version: 1.0 Message-Id: In-Reply-To: References: Date: Mon, 11 Apr 2005 16:35:25 -0700 To: Paxton , From: Brent Chapman Subject: Re: available network automation tools Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Archive-Number: 200504/23 X-Sequence-Number: 23 At 8:19 AM -0700 4/11/05, Paxton wrote: >on network modeling, have you all come up with your own model or used >something existing? Or taken something existing and modified it? > >I have looked at CIM and after scratching my head for a few days finally >realized that I think of the network in terms of associations, and CIM is >laid out by generalizations (inheritance.) The associations are defined, >but are sort of hard to find because of the generalization perspective. I >don't want to get into a semantics discussion of what's right, I'm more >interested in being able to hand a database to a network engineer and say: >go for it. IMO a db schema laid out with CIM is too confusing because it >isn't laid out according to associations, which is (IMO) how network >engineers (vs IT modeling guys) see things. Can you point us to any references for CIM? I'm not familiar with it. >Anyway, I have done this before and we sort of took some clues from what >was available at the time (SNMP and some vendor tools.) It wasn't exactly >right, but I learned a lot from the experience. We put together the >database for one specific tool, then realized afterwards how much else it >could be used for. I believe the core piece missing is the network >model/database, consider all the other parts (how to update devices, >monitoring configuration changes, etc) as component services that the >database enables. Not all the component parts have to be written for the >database to have value, and not everyone will want to use the same >component parts. Bingo! I've been thinking much the same thing... >Are you all interested in starting an open source project around this? I don't know about anybody else, but I certainly am. I think that such a "network database" would then become a platform upon which much else could and would be built; like you said, though, it's currently the core missing piece. -Brent -- Brent Chapman -- Great Circle Associates, Inc. Specializing in network infrastructure for Silicon Valley since 1989 For info about us and our services, please see http://www.greatcircle.com/ Network Automation blog: http://www.greatcircle.com/blog/network_automation From network-automation-owner@greatcircle.com Mon Apr 11 16:37:28 2005 X-Original-To: network-automation@greatcircle.com Received: from zproxy.gmail.com (zproxy.gmail.com [64.233.162.200]) by mycroft.greatcircle.com (Postfix) with ESMTP id 48EBF32C4DE for ; Mon, 11 Apr 2005 16:37:27 -0700 (PDT) Received: by zproxy.gmail.com with SMTP id 12so480899nzp for ; Mon, 11 Apr 2005 16:37:26 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=TdErcJEgpjw+8d8KtouvSCQvJ0GR8YkIIi1qsybo4K1uQxknwDtC4SswLDf+batQd45sCouHiREQN6A4XlDs664t5zlSdaZiwo1pXRP4aI/Badsm0er7pUo4LdObd53EzNj0QbnYR4SBWsJo48TaMdOvOoIalGDJizdWO/qVwnA= Received: by 10.36.80.17 with SMTP id d17mr218327nzb; Mon, 11 Apr 2005 16:37:25 -0700 (PDT) Received: by 10.36.67.1 with HTTP; Mon, 11 Apr 2005 16:37:25 -0700 (PDT) Message-ID: <18f6019405041116371c27f131@mail.gmail.com> Date: Mon, 11 Apr 2005 16:37:25 -0700 From: Aaron Glenn Reply-To: Aaron Glenn To: Brent Chapman Subject: Re: available network automation tools Cc: network-automation@greatcircle.com In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> <42593FE4.7030901@gmail.com> X-Archive-Number: 200504/24 X-Sequence-Number: 24 On Apr 11, 2005 4:30 PM, Brent Chapman wrote: > > Yes, this is important; it's got to be extendable, because you aren't > going to think of everything up front. > Can one model a network in such a way ontop of SQL, or is there a better (easier/more extensible/effecient) way? aaron.glenn From network-automation-owner@greatcircle.com Mon Apr 11 16:42:09 2005 X-Original-To: network-automation@greatcircle.com Received: from [66.92.48.19] (localhost [127.0.0.1]) by mycroft.greatcircle.com (Postfix) with ESMTP id CD23B32C4E1; Mon, 11 Apr 2005 16:42:08 -0700 (PDT) Mime-Version: 1.0 Message-Id: In-Reply-To: <18f6019405041116371c27f131@mail.gmail.com> References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> <42593FE4.7030901@gmail.com> <18f6019405041116371c27f131@mail.gmail.com> Date: Mon, 11 Apr 2005 16:42:05 -0700 To: Aaron Glenn From: Brent Chapman Subject: Re: available network automation tools Cc: network-automation@greatcircle.com Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Archive-Number: 200504/25 X-Sequence-Number: 25 At 4:37 PM -0700 4/11/05, Aaron Glenn wrote: >On Apr 11, 2005 4:30 PM, Brent Chapman wrote: >> >> Yes, this is important; it's got to be extendable, because you aren't >> going to think of everything up front. >> > >Can one model a network in such a way ontop of SQL, or is there a >better (easier/more extensible/effecient) way? I don't know. It would seem to me that being _able_ to represent your network model in SQL is a worthwhile goal, but I'm not sure if the constraints imposed by SQL would force too many compromises in the model. I'd be tempted to model the data structures and relationships in the abstract, and then figure out how to represent those in SQL (or any other tool, including plain text). -Brent -- Brent Chapman -- Great Circle Associates, Inc. Specializing in network infrastructure for Silicon Valley since 1989 For info about us and our services, please see http://www.greatcircle.com/ Network Automation blog: http://www.greatcircle.com/blog/network_automation From network-automation-owner@greatcircle.com Mon Apr 11 16:45:34 2005 X-Original-To: network-automation@greatcircle.com Received: from darksun.binsh.com (darksun.doomathon.com [64.232.216.23]) by mycroft.greatcircle.com (Postfix) with ESMTP id 6B78732C4A9; Mon, 11 Apr 2005 16:45:33 -0700 (PDT) Received: from darksun.binsh.com (localhost [127.0.0.1]) by darksun.binsh.com (8.12.6/8.12.6) with ESMTP id j3BNkK5C000253; Mon, 11 Apr 2005 16:46:20 -0700 (PDT) Received: from localhost (paxton@localhost) by darksun.binsh.com (8.12.6/8.12.6/Submit) with ESMTP id j3BNkJqE000250; Mon, 11 Apr 2005 16:46:19 -0700 (PDT) X-Authentication-Warning: darksun.binsh.com: paxton owned process doing -bs Date: Mon, 11 Apr 2005 16:46:19 -0700 (PDT) From: Paxton To: Brent Chapman Cc: Subject: Re: available network automation tools In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Archive-Number: 200504/26 X-Sequence-Number: 26 sure thing... http://www.dmtf.org/standards/cim/ I have heard that cisco makes use of cim in something they are doing, possibly this: http://www.cisco.com/en/US/products/sw/netmgtsw/ps4748/products_programming_usage_guide_chapter09186a00801e18e6.html but I don't know for sure. CIM has also been debated in the netconf discussions: http://www.ietf.org/html.charters/netconf-charter.html On Mon, 11 Apr 2005, Brent Chapman wrote: > At 8:19 AM -0700 4/11/05, Paxton wrote: > >on network modeling, have you all come up with your own model or used > >something existing? Or taken something existing and modified it? > > > >I have looked at CIM and after scratching my head for a few days finally > >realized that I think of the network in terms of associations, and CIM is > >laid out by generalizations (inheritance.) The associations are defined, > >but are sort of hard to find because of the generalization perspective. I > >don't want to get into a semantics discussion of what's right, I'm more > >interested in being able to hand a database to a network engineer and say: > >go for it. IMO a db schema laid out with CIM is too confusing because it > >isn't laid out according to associations, which is (IMO) how network > >engineers (vs IT modeling guys) see things. > > Can you point us to any references for CIM? I'm not familiar with it. > > >Anyway, I have done this before and we sort of took some clues from what > >was available at the time (SNMP and some vendor tools.) It wasn't exactly > >right, but I learned a lot from the experience. We put together the > >database for one specific tool, then realized afterwards how much else it > >could be used for. I believe the core piece missing is the network > >model/database, consider all the other parts (how to update devices, > >monitoring configuration changes, etc) as component services that the > >database enables. Not all the component parts have to be written for the > >database to have value, and not everyone will want to use the same > >component parts. > > Bingo! I've been thinking much the same thing... > > >Are you all interested in starting an open source project around this? > > I don't know about anybody else, but I certainly am. I think that > such a "network database" would then become a platform upon which > much else could and would be built; like you said, though, it's > currently the core missing piece. > > > -Brent > -- > Brent Chapman -- Great Circle Associates, Inc. > Specializing in network infrastructure for Silicon Valley since 1989 > For info about us and our services, please see http://www.greatcircle.com/ > Network Automation blog: http://www.greatcircle.com/blog/network_automation > From network-automation-owner@greatcircle.com Mon Apr 11 18:05:53 2005 X-Original-To: network-automation@greatcircle.com Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.204]) by mycroft.greatcircle.com (Postfix) with ESMTP id 7592B32C4DB for ; Mon, 11 Apr 2005 18:05:52 -0700 (PDT) Received: by rproxy.gmail.com with SMTP id r35so1357587rna for ; Mon, 11 Apr 2005 18:05:51 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=YYj1lQg7xAcgTH9pJqpzppOBd0EPvR70ZW2xg5g75klJqp/QwmSWFpcW+OQxMhEgmLhqqyJJegUUNHPHN3jIBgK7qxBTiXuD97oiWUWTszJ0tZb9VUwkxXmSp2CDuBAKlwAiVNd5ftcxM6gY/ycjyjPnFVXJyk8w+RhL4Y5avsA= Received: by 10.38.90.49 with SMTP id n49mr370903rnb; Mon, 11 Apr 2005 18:05:51 -0700 (PDT) Received: by 10.39.1.43 with HTTP; Mon, 11 Apr 2005 18:05:51 -0700 (PDT) Message-ID: <7654d9d05041118057c4ed5fa@mail.gmail.com> Date: Tue, 12 Apr 2005 11:05:51 +1000 From: Andrew Fort Reply-To: Andrew Fort To: Daniel Hagerty Subject: Re: available network automation tools Cc: network-automation@greatcircle.com In-Reply-To: <16986.6174.403374.979347@perdition.linnaean.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> <42593FE4.7030901@gmail.com> <7654d9d05041021497c88c860@mail.gmail.com> <16986.6174.403374.979347@perdition.linnaean.org> X-Archive-Number: 200504/27 X-Sequence-Number: 27 On Apr 11, 2005 4:24 PM, Daniel Hagerty wrote: > I'm not sure how tall we'll have to climb to really produce > satisfactory reasoning with the problem of "this" yet. And that's the heart of it. Routing protocols suggest a common "this" (the IGP configuration), and the mixing of syntax and semantics got me re-reading "godel, escher, bach". > The IGP example, the destroy the ip address of the interface, and many > other complex "godel" looking problems in configuration come down to > the presence of an implied "this" concept. For simple things like > "this router", it's can be easy enough to express. Other things are > probably less so. Our approach is inherently "network" specific (i mean, "my network" versus "your network"), because the semantics of this network are tied up in the syntax of our language of abstraction. When you solve the problem, some of the lessons learned may be useful for me, but some will undoubtedly be overly simplistic or not complex enough to avoid unwanted looping or halting states. Beauty in the world is such a difficult thing when you just want to automate your network! -andrew From network-automation-owner@greatcircle.com Mon Apr 11 20:29:46 2005 X-Original-To: network-automation@greatcircle.com Received: from perdition.linnaean.org (dsl092-073-217.bos1.dsl.speakeasy.net [66.92.73.217]) by mycroft.greatcircle.com (Postfix) with ESMTP id 3A33532C338; Mon, 11 Apr 2005 20:29:45 -0700 (PDT) Received: by perdition.linnaean.org (Postfix, from userid 31013) id D15FF19344; Mon, 11 Apr 2005 23:29:35 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16987.16543.735569.79861@perdition.linnaean.org> Date: Mon, 11 Apr 2005 23:29:35 -0400 To: Brent Chapman Cc: Aaron Glenn , network-automation@greatcircle.com Subject: Re: available network automation tools In-Reply-To: References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> <42593FE4.7030901@gmail.com> <18f6019405041116371c27f131@mail.gmail.com> X-Mailer: VM 7.19 under Emacs 20.7.1 Reply-To: Daniel Hagerty From: Daniel Hagerty X-Archive-Number: 200504/28 X-Sequence-Number: 28 > I'd be tempted to model the data structures and relationships in the > abstract, and then figure out how to represent those in SQL (or any > other tool, including plain text). Your temptation is good intuition; listen to it. Producing something concrete like a SQL model is effectively syntax -- something you never want to do first. Producing the model and using it to produce represenations from it (be they text, sql, or whatever) works better. Also, this approach allows you to leverage the strenghts of each tool as appropriate; if your model can be represented either as text or SQL, you can presumbably convert between the two forms as needed. For example, SQL is good for structured search, whereas text can be better for a mostly undirected browse. From network-automation-owner@greatcircle.com Mon Apr 11 21:09:36 2005 X-Original-To: network-automation@greatcircle.com Received: from darksun.binsh.com (darksun.doomathon.com [64.232.216.23]) by mycroft.greatcircle.com (Postfix) with ESMTP id EBCB732C2C2 for ; Mon, 11 Apr 2005 21:09:26 -0700 (PDT) Received: from darksun.binsh.com (localhost [127.0.0.1]) by darksun.binsh.com (8.12.6/8.12.6) with ESMTP id j3C4AC5C000469; Mon, 11 Apr 2005 21:10:12 -0700 (PDT) Received: from localhost (paxton@localhost) by darksun.binsh.com (8.12.6/8.12.6/Submit) with ESMTP id j3C4ABdP000464; Mon, 11 Apr 2005 21:10:12 -0700 (PDT) X-Authentication-Warning: darksun.binsh.com: paxton owned process doing -bs Date: Mon, 11 Apr 2005 21:10:11 -0700 (PDT) From: Paxton To: "Network.Security" Cc: Subject: Re: available network automation tools In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Archive-Number: 200504/29 X-Sequence-Number: 29 Scott - do you guys use this as well: http://www.opnet.com/products/itsentinel/ITSentinel.pdf I noticed you are from target and they list target as a customer. from their site it looks like they started out as a predictive analysis sort of thing, but also have some config mgt, possibly added later??? If you do use it, can you comment on it? thanks! On Fri, 8 Apr 2005, Network.Security wrote: > OK, I'll bite. > > We use Rendition (now Opsware) for config mgmt. of our network stuff and > the change detection / archival function has already saved my butt a > number of times. Our engineers gripe about the Big-Brother aspect, but > per the quasi-rant about SOX / CISP / PCI it's a fact of life now for > any SEC-filing corp at a minimum. > > I realize this is somewhat OT for this list, but SOX (in general) > shouldn't really matter to network admins (network meaning L1 to ~L4), > as SOX is all about altering the financial data. As a network person, > you can certainly see all of that data, but you can't change it (packet > injection doesn't count, most apps pick up on that sort of activity even > if they don't know why they know :) For that you need to be a server > admin / DBA, so for true network people, we don't really care. (Loose > generalization there. Re: caring) > > As for Opsware, their SOX report is static. It doesn't tell you > anything. It's just text so the tool offers no value to that. > > Now, CISP / PCI on the other hand, that's the project funding behemoth > you've all been waiting for. If you need money, say it's for PCI and > poof-like-magic, here's the cash to make it happen. PCI has some fairly > strict requirements that are defined to the network level regarding open > ports, encryption schemes, use of clear-text, etc. Tools like Opsware > can help enforce or at least notify on those data points. > > Re: free tools, I think we've all heard of RANCID as a config-o-monitor > (I personally am CVS debilitated and have not yet been able to make it > work on any platform). Big companies do not like free tools. That's > why Linux was not making progress in large enterprises until we could > start paying for it (aka Red Hat). If they can't get maintenance / > support for something that the business needs, it's not coming into the > environment. This makes sense, though not really from a "help us stay > on the cutting-edge of technology aspect". > > Opsware and Voyence (we demo-ed them) both do some configuration > templating so that if you are building out CPE, you can have it make > your router configs or whatever, but for already installed networks, the > templating is not valuable. I'm all about the configuration policy > piece, I want to know how many of my devices don't have enough ntp > servers configured, that sort of thing or down the road to make sure my > QoS policies are consistent across the board. But that's all to get me > to a point in time where everything is "right", it doesn't help me > deploy new services all that differently than my perl scripts did > before. > > What I think you are talking about is an application aware network > provisioning system. Something that is aware of all possible > topological paths between endpoints and is smart enough to know how to > configure all the hops / connections in between the two to make > something happen. Like a DOS mitigation system or punching holes > through a series of firewalls or configuring multi-hop VPN tunnels. > Yeah, that doesn't exist as far as I know. Or more to the point, I'm > sure these tools could do that, but the work required on the front-end > won't end up saving you anything on the back-end. Oh and it has to be > vendor-agnostic. Heh. > > That market is more in the NetDoctor or related simulation style > configuration analyzers that do the what-if type stuff or there are a > couple other QoS policers out there that do something similar, but they > are niche market tools for QoS only and appliances at that. > > So to your question, No would be my answer, there isn't something like > that out there today, there are point solutions that solve particular > aspects of the overall issue, but nothing end-to-end. > > BTW, Opsware and others would love to see this list if it grows and > utilize it as a tool for soliciting industry-wide customer feedback. I > would suggest consideration of that either for it or against (I don't > really care) and noting it in the policy. > > Scott.altman@target.com > > From network-automation-owner@greatcircle.com Mon Apr 11 22:18:12 2005 X-Original-To: network-automation@greatcircle.com Received: from perdition.linnaean.org (dsl092-073-217.bos1.dsl.speakeasy.net [66.92.73.217]) by mycroft.greatcircle.com (Postfix) with ESMTP id A51A232C50C for ; Mon, 11 Apr 2005 22:18:11 -0700 (PDT) Received: by perdition.linnaean.org (Postfix, from userid 31013) id 335B219344; Tue, 12 Apr 2005 01:18:02 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16987.23050.77047.3526@perdition.linnaean.org> Date: Tue, 12 Apr 2005 01:18:02 -0400 To: Andrew Fort Cc: network-automation@greatcircle.com Subject: Re: available network automation tools In-Reply-To: <7654d9d05041118057c4ed5fa@mail.gmail.com> References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> <42593FE4.7030901@gmail.com> <7654d9d05041021497c88c860@mail.gmail.com> <16986.6174.403374.979347@perdition.linnaean.org> <7654d9d05041118057c4ed5fa@mail.gmail.com> X-Mailer: VM 7.19 under Emacs 20.7.1 Reply-To: Daniel Hagerty From: Daniel Hagerty X-Archive-Number: 200504/30 X-Sequence-Number: 30 > Our approach is inherently "network" specific (i mean, "my network" > versus "your network"), because the semantics of this network are tied > up in the syntax of our language of abstraction. When you solve the > problem, some of the lessons learned may be useful for me, but some > will undoubtedly be overly simplistic or not complex enough to avoid > unwanted looping or halting states. You seem to misunderstand the actual scope of my hammer, but then, I haven't shown you my hammer, so this is obviously my error. I'm not actually coming at this from network configuration land; I'm coming from some place that is somewhat harder in one key aspect: non-commodity software staging. Networks don't change vocabulary as often as bleeding edge new software does. This change stresses configuration generation concepts enormously. I had to go back to the blackboard for a little bit in order to think about what's going on with software staging. But anyway... What I would like to give you should have "well formed" semantics where termination of the base cases is promised by the semantic. If I can't deliver on this, I have nothing. Given such a promise, nothing prevents you from mis-expressing your domain as non-terminating in all or only some conditions. Programming is programming, as always. I would like to give you all is a means of description of semantics and knowledge "issues". Solutions to my own problem requires such a means. However, keep in mind that my code is a very small thing; it isolates a key pattern that will always be, but your problem is still your own. I just give you a framework for the common pattern your problem always falls into and provide means to relate how your program interacts with the broader world. As network operators, you want something that deals in your own domain; this code is not in that domain. You make the domain of networks out of other domains, all of which eventually come down to the domain of domains. What I have in hand is something like a recursion combinator. It's not a recursion combinator, in that a recursion combinator typically enables recursion and no more. This combinator mixes the recursion concept with a multiplexing concept. Logical "today"'s project (which will probably not happen today) is to produce a pair of mutually recursive functions that express this mutual recursion within my framework. Producing this is likely to exercise the multiplexing portion of the combinator in some fashion or another. The only example I've had time to write at this layer only exercises recursion. Obviously, this seems to have little to do with "can I program my cisco router with that?". Like I said, there was a bit of going back to the blackboard involved. Code, examples, prior art refrences, etc, etc available upon request, as long as you keep in mind that it's all a work in progress, and it's very early in the game. I'll mention that asking me "how would you reason with...?" sounds like a fun game to play, and would be helpful for me in finding many people's different examples. From network-automation-owner@greatcircle.com Tue Apr 12 01:20:59 2005 X-Original-To: network-automation@greatcircle.com Received: from gate.epygi.de (gate.epygi.de [212.126.211.241]) by mycroft.greatcircle.com (Postfix) with ESMTP id D20A332C514 for ; Tue, 12 Apr 2005 01:19:16 -0700 (PDT) Received: from NBGIO2 (nb-gio2.epygi.de [10.20.0.21]) by gate.epygi.de (8.13.3/8.13.3) with ESMTP id j3C8K73V021334 for ; Tue, 12 Apr 2005 10:20:09 +0200 (METDST) From: "Georg Magschok" To: Subject: Re: available network automation tools Date: Tue, 12 Apr 2005 10:22:45 +0200 Organization: Epygi Labs DE GmbH Message-ID: <002101c53f38$d186d480$1500140a@EPYGI.DE> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.6626 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 In-Reply-To: Importance: Normal X-Archive-Number: 200504/31 X-Sequence-Number: 31 > >Are you all interested in starting an open source project around this? > I don't know about anybody else, but I certainly am. I think that > such a "network database" would then become a platform upon which > much else could and would be built; like you said, though, it's > currently the core missing piece. Coming from network management development for ATM switches and VoIP boxes, I'd like to share an idea that I'm going to implement sooner or later and would like to see approved as "not wrong": "The semantics of a network model should be defined by the network elements!" The reason is rather simple: Either a network management system needs an awful amount of information about a network that it should work on (and this would either need to be standardized into the last little detail or manually entered by admins), or the network must be able to provide information about itself. Currently I'm merely waiting for a standard to support that approach, or a few global players to start using something like it. Examples: a) the stone-age SNMP approach to this: a private MIB is generated from within the sources of device-firmware and distributed by a device to its managers to be compiled and give the admin access to configuration/accounting/security/fault/performance information that exceeds the public MIBs. [no semantics in this approach other than the textual decriptions of Managed Objects] b) in the XML-world the managed entity would be capable of producing a Schema for it's management data (Preferrably precompiled, but on strong, complicated network elements it may be dynamically generated). I hoped that Netconf or CIM would sooner or later define this. Transportation using the Netconf drafts is fine, though. (Yes, NetML could be a syntax to use for this) c) web based management is already quite near this approach, (because it keeps much of the intelligence for device control inside of the devices) but it has far too many proprietary variants, and automating it is messy. (maybe WBEM could help here, but I didn't look into that so far) In other words: I believe that trying to build management software that tries to manage everything is fine for today's applications, but for the future it would be easier to handle this by offloading information from that management software to the devices/elements/CPE, using a standardized approach. Can anybody comment if this is completely stupid or has been done already or may be a useful approach? Thanks, Gio Epygi Labs DE GmbH - Georg 'Gio' Magschok From network-automation-owner@greatcircle.com Tue Apr 12 01:24:11 2005 X-Original-To: network-automation@greatcircle.com Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.202]) by mycroft.greatcircle.com (Postfix) with ESMTP id 8FCAB32C351 for ; Tue, 12 Apr 2005 01:23:02 -0700 (PDT) Received: by rproxy.gmail.com with SMTP id a41so1460846rng for ; Tue, 12 Apr 2005 01:24:07 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=GA3TtKk1FlcjVClAz6mo2GBqK5S13XhD6P1pnjrMUELqePaaPTTawDPcoewcCtTIrYRpp9Z5miCEeV9w3UMThVbyv/yncOyjlABw4ZBk+wFHORDBeKo/SqldvUvXGNgRWUatl1Bw1cklE+yd36QH+6zF7SLB9YaIn3yXNmv/Zrw= Received: by 10.38.153.43 with SMTP id a43mr4774311rne; Tue, 12 Apr 2005 01:24:06 -0700 (PDT) Received: by 10.39.1.43 with HTTP; Tue, 12 Apr 2005 01:24:06 -0700 (PDT) Message-ID: <7654d9d05041201245c28db72@mail.gmail.com> Date: Tue, 12 Apr 2005 18:24:06 +1000 From: Andrew Fort Reply-To: Andrew Fort To: Daniel Hagerty Subject: Re: available network automation tools Cc: network-automation@greatcircle.com In-Reply-To: <16987.23050.77047.3526@perdition.linnaean.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> <42593FE4.7030901@gmail.com> <7654d9d05041021497c88c860@mail.gmail.com> <16986.6174.403374.979347@perdition.linnaean.org> <7654d9d05041118057c4ed5fa@mail.gmail.com> <16987.23050.77047.3526@perdition.linnaean.org> X-Archive-Number: 200504/32 X-Sequence-Number: 32 On Apr 12, 2005 3:18 PM, Daniel Hagerty wrote: > Code, examples, prior art refrences, etc, etc available upon > request, as long as you keep in mind that it's all a work in progress, > and it's very early in the game. Prior art sounds good. I've really been enjoying some of the reading (as well as your insight) in this area lately, and I can always add to my reading list. -andrew From network-automation-owner@greatcircle.com Tue Apr 12 05:26:29 2005 X-Original-To: network-automation@greatcircle.com Received: from mail-white.research.att.com (mail-red.research.att.com [192.20.225.110]) by mycroft.greatcircle.com (Postfix) with ESMTP id 9C15132C54C for ; Tue, 12 Apr 2005 05:26:28 -0700 (PDT) Received: from light.research.att.com (light.research.att.com [135.207.27.144]) by bigmail.research.att.com (8.11.6+Sun/8.11.6) with ESMTP id j3CCQ9320847; Tue, 12 Apr 2005 08:26:10 -0400 (EDT) Date: Tue, 12 Apr 2005 08:26:09 -0400 From: Albert Greenberg To: Georg Magschok Cc: network-automation@greatcircle.com Subject: Re: available network automation tools In-Reply-To: <002101c53f38$d186d480$1500140a@EPYGI.DE> Message-ID: References: <002101c53f38$d186d480$1500140a@EPYGI.DE> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Archive-Number: 200504/33 X-Sequence-Number: 33 On Tue, 12 Apr 2005, Georg Magschok wrote: > > Coming from network management development for ATM switches and VoIP > boxes, I'd like to share an idea that I'm going to implement sooner or > later and would like to see approved as "not wrong": > > "The semantics of a network model should be defined by the network > elements!" > I think you are right, though the level of abstraction providedd by the network elements may be quite low if you want network-wide rather than box by box management. For example, ACLs can provide network-wide perimeter defense, with appropriate clauses embedded in the edge routers. It's all there in the routers, but it is not expressed directly as a network-wide constraint (which might help manage ACLs). Albert Greenberg; AT&T Labs-Research; Florham Park, NJ 07932-0971 From network-automation-owner@greatcircle.com Tue Apr 12 05:59:30 2005 X-Original-To: network-automation@greatcircle.com Received: from gate.epygi.de (gate.epygi.de [212.126.211.241]) by mycroft.greatcircle.com (Postfix) with ESMTP id 04C0432C357 for ; Tue, 12 Apr 2005 05:59:29 -0700 (PDT) Received: from NBGIO2 (nb-gio2.epygi.de [10.20.0.21]) by gate.epygi.de (8.13.3/8.13.3) with ESMTP id j3CCxRW3022807; Tue, 12 Apr 2005 14:59:27 +0200 (METDST) From: "Georg Magschok" To: "'Albert Greenberg'" Cc: Subject: Re: available network automation tools Date: Tue, 12 Apr 2005 15:02:04 +0200 Organization: Epygi Labs DE GmbH Message-ID: <002601c53f5f$d62f6340$1500140a@EPYGI.DE> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.6626 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 In-Reply-To: Importance: Normal X-Archive-Number: 200504/34 X-Sequence-Number: 34 > > "The semantics of a network model should be defined by the network > > elements!" > I think you are right, though the level of abstraction providedd by the > network elements may be quite low if you want network-wide rather than box > by box management. For example, ACLs can provide network-wide perimeter > defense, with appropriate clauses embedded in the edge routers. It's all > there in the routers, but it is not expressed directly as a network-wide > constraint (which might help manage ACLs). The node would know well about itself, but not too much about the network other than it's direct box by box relations or being part of various kinds of peer groups. The network-wide constraints are not solved by the approach I propose, only supported. Thank you! Gio Epygi Labs DE GmbH - Georg 'Gio' Magschok From network-automation-owner@greatcircle.com Tue Apr 12 07:23:18 2005 X-Original-To: network-automation@greatcircle.com Received: from mtxexch01.add0.masergy.com (unknown [64.47.104.21]) by mycroft.greatcircle.com (Postfix) with ESMTP id 2346732C2BC; Tue, 12 Apr 2005 07:23:11 -0700 (PDT) Received: from [64.47.15.109] ([64.47.15.109]) by mtxexch01.add0.masergy.com with Microsoft SMTPSVC(6.0.3790.211); Tue, 12 Apr 2005 09:22:30 -0500 Message-ID: <425BD965.5020306@gmail.com> Date: Tue, 12 Apr 2005 10:21:25 -0400 From: Kirby Files User-Agent: Mozilla Thunderbird 1.0.1 (X11/20050309) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Brent Chapman Cc: Aaron Glenn , network-automation@greatcircle.com Subject: Re: available network automation tools References: <42573546.7050503@gmail.com> <18f60194050409174575ada600@mail.gmail.com> <42593FE4.7030901@gmail.com> In-Reply-To: