Also, having VOIP on your internet line is KILLING your bandwidth I bet, especially with 700+ms.... The way VOIP is, the VOIP packets get the highest priority, so all other traffic is delayed when the phone is in use... might want to look into upgrading your internet speed.
I couldn't follow the description of your network... See if you can use MS Paint, and make a small .jpg file of your network topology, with boxes and lines... the boxes being the hardware devices (with hardware model numbers_ and the lines being the wires....