Forums

Please or Register to create posts and topics.

Latency issues: Fuck em!

Hello!

We have had some WAN issues and need diagnostic information to help track down the cause! For 30-60 seconds, roughly 1/3rd of the pop has had latency increase enormously, peaking around 6,000! To help isolate the issue, it would be helpful if people experiencing the issue could supply me with some diagnostic information.  Want to help? Start by downloading & staging WinMTR

1)      Browse to http://winmtr.net/download-winmtr/

2)      Download the appropriate version (probably WinMTR 64 bit for Windows 64 )

3)      Right click the file, select “Exctract all”, then “Extract”

4)      Browse to and launch the respective .exe; for me this is "C:\Users\Amarsaudon\Downloads\WinMTR-v092\WinMTR-v092\WinMTR_x64\WinMTR.exe"

5)      In WinMTR, enter Host: rust.rustorbust.org

6)      Say “I WILL HELP YOU ADMIN FUCKWAD” in chat, so I will add your IP (I monitor the same data in reverse from the server during a spike)

And now we wait. If a spike hits:

1)      Start bitching! Say “LAAAAAG” in chat (Alerts me to the spikes!)

2)      Click “Start”

3)      After 4-5 minutes (once the spike has settled) hit “Stop”, “Copy Text to clipboard”, Paste into an email to alex@rustorbust.org and hit send!

4)      Profit!

Alright folks, here is a sample output from a client experiencing the issue:

 

rust.txt - Notepad

Notice the loss percentages starting with 64.125.31.86 IP, encompassing the bulk of the zayo routers. Issues begin external of my host linode's network, with Zayo Networks being the most likely culprits!

Unfortunately they are external if my host (linode), and my host's ISP. I have reported the issue, but ultimately have no idea when this might be addressed.

Here is the full server-to-client traces mid-spike:

http://pastebin.com/xRn4r4KS

Sorry about this issue, but thanks for your patience while Zayo sorts their shit out!

-Alex

Bonus: Pings exceeding 6,000: https://imgur.com/FWqyFJR

During this same period, over half the server was pinging under 50. Sorry to the poor soles that were hit with this.

 

So far only 3 spikes consistent with these symptoms have happened in the past 24 hours, only 1 in the past 12. Looks like this issue may be resolved!

Thanks to the people who kindly submitted me your MTR reports! These proved to be immensely valuable in troubleshooting this issue, and will certainly be part of my standard troubleshooting should latency issues arise down the road!

-Alex