PCA Forums

Well I need some help trying to narrow down and solve this problem so bear with me a min while explain what's going on ...

For 20 years we have had a network running in our building .. simple peer - peer network nothing fancy.

Up until recently we never really had any trouble with it at all. About 2 months ago we got all new computers (predone dual core X2s all the same from HP -- 9 machines total + one server also a dual core but built here with different components).

Things seemed fine and no problems that I am aware of. Within a week of getting the sytems I had Kaspersky Internet Security installed on all machines. About a month ago we started having SERIOUS network slowdown issues. Examples below

Loading of simple MS Word files from the server would take 5-10mins instead of 2 seconds.

Loading of certain programs from the server (our tax and accounting software runs via a network install over the server ALL other software runs locally) that used to take 5 seconds might now take 10-15 mins.

Transferring a small fil 100mb or so might have only been a few seconds now can take 30 mins.

This issue is VERY RANDOM. 1 or 2 computers on the network may have this issue but the others act normal. Then the next day the 1 or 2 systems previously affected now are fine.

Also often the slowdown problem can be eliminated by rebooting the affected computers several times (sometimes 5 or 6 times).

I have tried everything my feeble skills can think of including all of the following:

1. Rebooting the server when the slowdowns occur -- gives no help
2. Replaced all NICs with new ones - no help
3. Installed a new switch -- seemed to help and then a day later we were back to the same problems
4. Rebuilt the entire server with all new parts (motherboard, CPU, HDDs, NIC, Power supply etc).

Folks I am at a complete loss here and have no clue what to do but performance is so bad now that I have people waiting on the damn computers vs. actually working. Any help you guys can provide would be terrific.

Hardware/Setup notes

All computers are the same and have the same NICs installed (Intel 10/100/1000 PCI NICs).

NICs are all set to autonegotiate. I have tried hard setting them and that has no bearing on the performance

All wiring is standard cat5 copper wiring and is in tact as much as I can tell

Server is a 3800 X2 running 4 SATA drives (7200rpm Seagates). With the exception of tax and accounting software that runs from the server it is just for data storage and files are accessed from it.

All workstations are running windows XP pro with all the latest updates.

All workstations are running (server as well) Kaspersky Internet Security v 6.0 with all the latest updates.

Norton Utilites is installed on all workstations as well (but not Norton AV or anything else).

Workstations have background programs running other than Norton and Kaspersky

Network is a simple peer to peer Workgroup.


PLEASE HELP!!

Thanks in advance

Greg

I seem to remember a problem at work a few years ago, where one workstation flooded the network with data and slowed the entire network down to a crawl. Can you disable the workstations in groups to see if one of them is causing a problem?

I guess so but the slowdown is only on certain computers at a time (not all of them) and its very random; ie one day its computers 1 & 2 then we are fine for a day or 2 and then the next day its computer 4, 7, & 9 etc.

Completely random.

I can say that at one time or another every computer in the building has experienced this slow down but not all of them at the same time and pretty not more than about 3 computers at time seem slow .. the rest run fine.

Greg

That's going to be really tough to find. It's one thing if one computer is intermittently slowing down the entire network. It's harder if only certain parts of the network are slowing down intermittently. Maybe you can cap the speed on the network cards to see if this minimizes the impact of the slow downs. You'd think there would be a spike of network activity during the slow downs that would show up somewhere.

Have you tried running a group of the machines without the Kapersky Internet Security enabled?

Internet Security Suites have been known to cause random issues, particularly network-related. Since that is probably the only common bond between all the machines, it is worth a try for you.

yep and it seems to help for about an hour or so and then the problem will come back.

Greg

Was this disable it for a handful or all of them?

both.

First I tried disabling it on just the machines that were slow at that time, then rebooted and things seemed ok. Problem comes back several hours later or not until the next day ... sometimes on the same machines or on others.

Then I disabled it on all machines for about 3 days and for about a day and half everything was fine then bam it came back.

I have resorted to having the server rebooted every morning before anyone gets here and then each machine is rebooted until the slowdown on that one machine stops (sometimes 5 or 6 reboots, this morning my personal machine took 12 reboots and 3 server reboots before I could access anything with any speed whatsoever).

Its definitely the most baffling problem I have ever been faced with in computing.

Greg

the problem is a hub
somewhere on your network u either have a hub(not a switch) or a print server that is malfunctioning

the problem u have described is exactly to a T what has happend on a customers network

well I wish that was the case bet there are no hubs on our network anywhere.

just 1 switch and a SNAP VPN router - thats all

Greg

You either have a problem with the switch (have your rebooted it? Switches are small computers and their brains do get scrambled) or there's a device on the network (computer, network printer, wifi access point) causing the problem. Could also be the router - it may be mis-handling WAN transmissions and flooding the LAN.

Could be a problem with the server. Have you tried to verify that it only happens with file transfers to the server?

Have you started using any new app based on the server that might be a bandwidth hog? Especially apps that have the executable on the server? I have a client that started using an accounting app that runs off an executable on the server AND the damn data files are HUGE. When they ran payroll it would take all day to print checks and it would slow the network to a crawl. I wound up having to buy a managed switch with a two 1Gbit port on it and connected the server and payroll PC to the 1Gbit ports. That solved the problem.

One bad cable somewhere could also be causing it - particularly between the switch and the server.

Someone may be hogging the network - downloading some music or videos, or running a service that's a network hog. If some idiot is downloading from bittorrent or running a badly behaved P2P app it could easily flood the network.

RECOMMENDATIONS:

- First off you should NOT be running any internet security app on a PC within an office LAN. It'll cause you nothing but grief. Just put a plain jane anti-virus app. You don't want anything else, and disable all firewalls. Stick AVG free version on all of them if neccesary (yeah violates AVG's EULA, but people do it anyway). Don't think this is your problem, but you need to eliminate it.

- Power cycle (power off - wait 10 seconds - power) the network switch to clear it's memory.

- Power cycle the Router.

- I would borrow or buy another switch (never hurts to have a spare) and swap it out.

- Try disconnecting the router next time it happens and see if it clears up (BTW you should be using static IPs on all the workstations so that if the router fails taking your DHCP server with it, it won't take down your network).

- Tedious (and not sure how big an office we're talking about), but after eliminating the above, next time it happens try either having everyone stop using their PCs for a few minute and watch the switch for excessive traffic on one port. That might indicate a PC or device that's flooding the network that's connected to that port.

My gut feeling is that your problem is the switch. Would explain your symptoms very nicely.

Hope this helps...

I can't add anything that FP or Sov haven't suggested, but I do agree that the switch is probably malfunctioning.

You don't have a spare switch on hand? You should always have a backup incase of failure to shorten downtime.

Well the problem is this is a new switch. I replaced the old because of this exact problem so I put a new one in place. For about 2 days we were fine then boom it started all over again.

Greg

Okay, so the switch is eliminated. You're down to the following in order of the most likely to least:

- bad router

- bad cable or jack somewhere (most likely between the switch and the server)

- some PC is flooding the network either due to a bad NIC, malware, virus, a user who's doing something that's a bandwidth hog like running a DC++ server or bittorrent, etc.

- some new app someone's running is a major network hog

Power off the router next time it happens and see if that fixes it.

If that doesn't do it, next try running a completely different cable to the server and avoid using the same jacks (easily done if the server is in the same room with the switch, harder if it's not - I carry around a 100' CAT-5 cable specifically for doing this sort of testing).

If that doesn't help then you need to disconnect each port on the switch one at a time while the problem is occurring to see if you can narrow it down to one PC that's causing the problem.

Shot in the dark: Any changes to the wiring lately? Re-routing of cables, changing patch cables, etc? A bad cable, or a cable running near a large electro-magnetic source (like a ballast transformer in a ceiling fluorescent light fixture - ALWAYS skirt cables around light fixtures - could be causing packet loss on the network).

I've seen real stupid sh!t like people plugging a PHONE into a network jack and screwing up the network.

Once I had one idiot once who connected his FM radio antenna to a network jack to get a better signal and caused all kinds of problems.

Hope this helps...