Game Freezing/ Lagging randomly

When playing ZE mostly (because of all the people / zombies)
my game freezes or seems to only progress like 2 FPS even tho its higher.
I've noticed with "net graph" that my "IN:" goes to like 3k-5k and "OUT:" is like 100.

I was wondering if anyone could help me to stop this?
Please note this is only happening to me, and everyone else is fine...
(I also limited my FPS to 60)

My CPU specs:
Nvidia GTX 580
AMD Phenom x6 2.8GhZ
Win XP SP3
4 GB Ram

Screenshot

It might be something server side or possible nvidia based as I'm running the gtx470 and getting a similar problem minus the info on the specs. But I've noticed other players reporting individualized lag spikes as well so I can't be to sure without knowing from them.

I'm not that much of a techie, but doesn't XP get flaky running on more than 2 cores with RAM exceeding 3.2GB? Not sure if it's related at all to your gameplay, but you're running a pretty beefy rig there for an XP OS.

Sounds like a your computers being throttled possibly lowering the CPU? , maybe you need to overclock/vault your computer , but it all depends on if this issue has been present before? , What computer Make/Model do you have?

Blarg on me. I keep thinking about 32-bit systems, and those are the ones that have issues pushing certain stats, at least from what I've been told. Running 64-bit shouldn't be an issue. Nevermind me, the silly one.