high network latency

Hi, everyone
A while ago I've noticed that we have quite a high latency in our network. Ping from a cable modem to CMTS gives me about 5-6 ms latency with random bursts to 15-20 ms. Is that normal? In a regular twisted pair LAN I've never seen anything above 1 ms even if the ping test was making it through a chain of cheap asian switches. Please, could anyone take a look what latency do you get in the network, are there bursts like in ours.

Thanks in advance

P.S. CMTS load is 40%. I've also tried the ping test in our test lab with a passive network. Latency is about 4-5 ms. There are way less bursts but still there are some.