Battles With Boredom

Switch Troubleshooting for Fun and Profit

I spent an entire day troubleshooting odd network problems at a local client last Friday. The symptoms were:

• Users could not
• get their email
• access files on the server
• log on successfully
• Internet access was sluggish, for some users, but fine for others.
• Software deployments from across the networks (content filtering software, Ghost images, GPO-deployed packages) would not deploy.
• Ping times from a workstation to a server (across 5 switches) ranged anywhere from <1ms to several thousand ms, or just plain timeouts.
• Ping times from a workstation to another workstation, connected on the same switch, were only marginally better, resulting in anything from 1ms to frequent timeouts and dropped packets.
• Ping times to the local switch, to which the servers were plugged in, ranged from 1ms to several hundred ms.

Several hours of troubleshooting resulted in the following highlights:

• After two hours of capturing packets, 30% of all network traffic was ARP. This is with only a handful of desktop machines powered on. Not good.
• Rebooting one switch made a huge difference, and all of a sudden network traffic was working again. However, things were still slow on occasion.

So tomorrow I will be making some VLANs to cut down on the broadcast traffic across these eight or so switches. Fun!