Hello,
I am attempting to troubleshoot a problem packet loss problem from a small regional ISP. What we see on our onsite router is packet loss intermittently in the 1-20% range. The cable modem is in bridge mode and we have a /30 subnet. We are receiving a very large (7000+ a min.) amount of ARP requests for IPs in other subnets that we do not own for instance 50.x.x.x/24 10.x.x.x/24 108.x.x.x/24. The source MAC address for all the ARP requests comes up as a CASA CMTS. Based on my research it looks like that the ISP is using Proxy ARP and Cable Bundling. Also, based on my research, it looks like CASA CMTS sends out an ARP request first before checking the DHCP server for the MAC address. Our local router has ARP table entries for 250-300 devices which all point to the CASA CMTS. I have a few questions:
1. I’m assuming the IP address of our default gateway is actually located on the Casa CMTS as opposed to the cable modem as we get the MAC address of the Casa CMTS due to cable bundling being enabled. If this is the case, why is the CMTS not filtering out ARP requests not on our subnet or at very least only forward broadcasts only on slave interfaces on a /24 of our subnet, do cable bundles forward all allowed broadcasts?
2. Would these ARP requests cause additional CPU load on the cable modem or does the modem just pass the traffic and not look at its ARP table to determine if it should respond? Would there be any other WAN interfaces on the modem that would need to respond to this traffic such as the diag interface of the modem (192.168.100.1) or some other interface?
3. Is this broadcast domain too large and what would be the best approach to take to talk to someone knowledgeable at the ISP to talk about the issue?
4. Our router is jumping between 30-70% memory usage and 10-50% CPU usage, while it looks like the hardware is powerful enough to handle looking at its ARP table 7000 a min, would it be worth putting a Cisco Switch in place with Static Arp inspection (ARP Storm Control) enabled and a manual ARP ACL to only respond to requests looking for IP addresses on our subnet?
1. It is likely that all of the IP space for all the cable modems and CPEs are located on 1-3 bundles on the CMTS. Each bundles constitute a mac broadcast domain. Think of each bundle as a vlan in a switch. When an arp request is made it is broadcast just like it would be in a l2 switch.
2. As you stated, the cable modem is a bridge, and as such does not look at any traffic that passes through it. Depending on whether a router is bundled with your modem, there could be router interfaces that would respond to arp requests, however, the fact that you see the arp requests on your router, means this is quite unlikely.
3. Without looking at the CMTS it would be difficult to tell whether the broadcast domain is too large. likely, one or many of the ISPs customers have viruses/malware that is trying to find other hosts to infect. If it tries to connect to a non-existant host, an arp request is made to the broadcast domain. There are ways to limit this traffic at the CMTS, such as arp throttles, no cable arp and dhcp source verify. But these would be ISP specific. Arp throttles are the easiest to implement and are usually standard practice, so this is where you should start talking to them about it.
4. I would try and work with the ISP first, let them know your issue, and see if they will help you with a solution. If not yes you could go to other extremes to solve the problem yourself.
Fortunately, small regional ISPs are generally easy to work with, and have much less hoops to jump through. When I worked for one, if a customer had a problem the phone guys couldn't solve they would come ask me and it would get solved. The likelihood of calling in and getting in touch with someone that can help you is quite high.
Its my understanding that the reason that you use cable bundling in conjunction with Proxy ARP is to reduce the waste of public IP addresses in a subnet. Why would they bundle disjointed subnets into one huge layer 2 broadcast domain? I would understand if we were only reviving ARP traffic destine for say 50.x.x.x/24 or even a /16, but why would we be receiving ARP traffic for a completely disjointed subnet such as 108.x.x.x/24? I am assuming they are on the same cable bundle, but is this a mis-configuration on the ISP's part or is there some logical reason to do this?
Sadly we have attempted to reach out to the ISP's level 2 tech support multiple times now and have not gotten any response. I will continue to push to see who we can get through to.
Its my understanding that the reason that you use cable bundling in conjunction with Proxy ARP is to reduce the waste of public IP addresses in a subnet. Why would they bundle disjointed subnets into one huge layer 2 broadcast domain? I would understand if we were only reviving ARP traffic destine for say 50.x.x.x/24 or even a /16, but why would we be receiving ARP traffic for a completely disjointed subnet such as 108.x.x.x/24? I am assuming they are on the same cable bundle, but is this a mis-configuration on the ISP's part or is there some logical reason to do this?
Sadly we have attempted to reach out to the ISP's level 2 tech support multiple times now and have not gotten any response. I will continue to push to see who we can get through to.