Hello all
I've been struggling with this for some time now, and thought I'd put it out to the hive mind and see what y'all are doing. We're trying to implement an automated system for dealing with the ever present issue of virus infected CPE equipment.
I'll start with a short-ish background of what we're doing and how it works, and then pose my question/ comments:
We admin a small Cable ISP. They run a Cisco uBR7223, and all the IP level stuff is based on Free and Open Source, Linux based software running on off the shelf gear. Some of it is modified to work and do what they need.
We run 3 pairs of redundant servers:
Pair 1 do all our provisioning: DHCP, TFTP, DNS, etc. The pair runs a homegrown provisioning system written in PHP/ MySQL that backends the DHCP server and dynamically rebuilds DHCP class/ config files when changes are detected. If anyone is interested in this, please contact me off list - it's Free and Open Source but a bear to set up :) We're planning on writing a module that wraps the docsis utility (docsis.sourceforge.net); currently we run docsis by hand as needed.
Pair 2 are our web/email servers: Sendmail, Courier IMAP (Backended by MySQL), Squirrel Mail, Apache2, mysql replication, etc.
Pair 3 is a straight forward pair of packet filtering systems.
All 3 server pairs run some combination of Heartbeat, drbd, rsync etc. to keep data replicated and safe, and provide failover capabilities.
Our problem is malicious traffic generated from customers' computers that have been infected with viruses/ Trojans/ spyware etc. and are port scanning/ spamming/ or otherwise misbehaving due to the users' incompetence/ ignorance/ laziness/ whatever.
We are too small to deploy and pay for Anti-virus software on all our users' computers. We don't want the support headache or expense and frankly, in my humble opinion, it's not our job: It's like holding the road crew responsible for keeping your car tuned up.
Anyway, here's what we're up to:
We want our firewalls to actively look for malicious traffic on the wire (Snort can do this), and trigger a response. Here's where I want to get tricky: Snort could simply drop all traffic from that IP and leave it at that, but that would just generate more helpdesk calls. I'd like to provide something a little more helpful to our users, while still stopping the potentially harmful traffic from leaving our network.
There are a number of ways I can envision to do this:
Snort could trigger a script or http POST/ Get to the provisioning server such that we could add a DHCP pool for virus infected hosts, and redirect all their traffic to a webserver that serves up a page with some helpful details regarding why that system can no longer get out to the 'net.
Snort could also trigger a combination of dropping packets and redirecting some (http) that would accomplish the above without knocking the system offline altogether. This would be the preferable way, because this would also help to address the ever annoying users who set their IP address manually. This method would also:
Cut down on angry helpdesk calls when people get their modems shut off for abuse
Allow people with infected systems to use another computer behind their modem to download updates/ patches etc.
Be much more proactive with shutting down infected systems... automatic and immediate is the goal here.
I also envision 'agents' to this system running on our SMTP servers, looking for similar issues and malicious traffic patterns (i.e. 1000 email attempts/ hour from a dynamic host) that would result in the same sort of trigger and response on the provisioning system. As infected computers become ever more prevalent, my sense of humour about infected systems continues to disappear :(
I have 2 questions related to the above goals:
1. I'd like to implement cable-source verify on our uBR. As I understand it, the ISC DHCP server now supports DHCPLEASEQUERY. Is anyone out there using this combination? Does it work? Is it stable and reliable?
2. There are times where it would be *really* helpful to be able to 'bounce' a customer modem from our web application (or from a script triggered by some even as outlined above). As I understand it, there are SNMP OIDs that can do this. Is anyone out there doing this, and would you be willing to share your OIDs/ MIBs, specifically for the uBR7223/ 7246vxr platforms?
In return, when we finally get this project done, we'll post it up on Sourceforge for all to use and benefit from. On that note, if anyone out there has PHP experience and would like to contribute, please let me know!
Thanks very much in advance,
Rubin
That's a great concept. Working at $BIGCABLECO, we manually bump modems into an abuse dhcp pool that provides a filtered cm config, an RFC1918 address and catch-all DNS to direct them to a page indicating why they were suspended. The biggest problem is that people complain they can't 'fix' their computer if it's got no net access. There's no auto-suspension based on port scans -- just abuse@ complaints. There's RIAA/MPAA/BSA, spam, and virus activities reasons that can be listed. The subscriber has to call an 800 number and talk to staff to get reactivated.
If I'd designed the system, I'd probably would have gone with assignment of a real world address, catch-all dns, and a filtered cm config. The redirect page they receive would explain why they arrived there and have links to live antivirus scanners and purchase sites like Window Live Onecare or Avast. The trick dns and cm config would have to be configured to allow access to the antivirus sites. The last section of the page would have a checkbox with something to the effect of "I have downloaded and installed antivirus software and believe the problem has been corrected". Once that's submitted, the modem could be reprovisioned and rebooted back to a normal state.
Anyway, I know of 'cable-source verify' working normally on other systems; I don't know the details on the dhcp server used. My own cmts boxen are not Cisco so I unfortunately can't help with the mibs -- modem reset is enterprise specific on mine. If you're lucky, you might find it at Cisco's site: http://tools.cisco.com/Support/SNMP/do/BrowseOID.do?local=en
You can browse or download mibs there.
I can answer number two for you....
the relevant oid is 1.3.6.1.4.1.9.9.116.1.3.1.1.8 info on it can be found here
Cisco's SNMP Navigator
the way this oid works is thus using snmpset
snmpset -v 2c -c community ubrIPorDNSname .1.3.6.1.4.1.9.9.116.1.3.1.1.8.0.2.138.211.124.110 i 1
the 0.2.138.211.124.110 is the mac address of the device in question encoded in decimal i.e.
0.2.138.211.124.110 = 0002.8ad3.7c6e
the type is integer and the value of 1 resets the device.
Also sounds like a good idea, I would be willing to help out with it would need to familiarize myself with snort first though.
Rubin,
I would be interested in seeing your software. I've been contemplating tearing into dhcpd to integrate mysql to do exactly what you have already done. Do you have a public download site?
I would be interested in seeing your software, since for now we make do with pure dhcp and hand written config files.