Unix: Better network connection insights with mtr

Traceroute is still a great tool, but mtr ("my traceroute") provides even more insights when you're looking into network routing problems.

The mtr tool -- "my traceroute", originally "Matt's traceroute" for the guy that first developed it (Matt Kimball in 1997) -- is in some ways like a combination of traceroute and ping and it provides a lot more data than the two of these commands combined.

Like ping and traceroute, mtr uses icmp packets to test connections. While traceroute is likely to be installed on every Unix system you use, you may have to separately install mtr. If you do, here are some commands for doing so:

  • Ubuntu or Debian systems: apt-get install mtr
  • Fedora or Centos: yum install mtr
  • Mac OS X: brew install mtr
  • FreeBSD: pkg install net/mtr

Like traceroute, mtr uses TTLs (time to live values) so that it can report on each leg of a route individually. It does this by setting the TTL to 1, then 2, then 3 and so on. Each time, it collects the round trip time for the next leg of the trip to the remote system. When it sets the TTL to 2, for example, it gets the timing information for the second leg. At each connection, the router decrements the TTL by one (this is what routers always do). At the final device, it becomes 0 and the tracing goes no further. Instead, an "ICMP TTL exceeded" event occurs just it has once at each of the other devices, measurements for the last device are sent back to the source and the report is done.

The latency (round trip) measurement is the timestamp when ICMP reply is received minus the timestamp when the probe was launched.

By default, traceroute issues three probes aper hop, thus you will see three numbers for each hop in the traceroute output.

Here is some sample traceroute output:

$ traceroute world.pts.com
traceroute to world.pts.com (192.74.137.5), 30 hops max, 40 byte packets
 1  pix (192.168.0.2)  0.255 ms  0.478 ms  0.443 ms
 2  * * *
 3  gig1-6.umcp-core.net.doz.org (136.160.255.33)  9.856 ms  9.343 ms  9.822 ms
 4  ten2-0.stpaul-core.net.doz.org (136.160.255.198)  3.401 ms  3.858 ms  3.681 ms
 5  te4-3.ccr01.bwi01.atlas.cogentco.com (38.104.12.17) 2.920 ms 2.859 ms 3.280 ms
 6  te4-2.ccr01.phl01.atlas.cogentco.com (154.54.2.174) 5.965 ms 5.945 ms 5.920 ms
 7  te0-0-0-7.ccr22.jfk02.atlas.cogentco.com (154.54.31.53) 9.084 ms te0-0-0-7.ccr21.
jfk02.atlas.cogentco.com (154.54.1.41)  8.811 ms te0-0-0-7.ccr22.jfk02.atlas.cogentco.
com (154.54.31.53)  8.784 ms
 8  be2096.ccr22.bos01.atlas.cogentco.com (154.54.30.42) 14.991 ms be2094.ccr21.bos01
.atlas.cogentco.com (154.54.30.14)  14.764 ms be2096.ccr22.bos01.atlas.cogentco.com 
(154.54.30.42)  14.964 ms
 9  te4-1.mag02.bos01.atlas.cogentco.com (154.54.43.70) 14.478 ms te4-1.mag01.bos01.
atlas.cogentco.com (154.54.43.50)  14.201 ms  14.171 ms
10  gi0-0-0-0.nr11.b000502-0.bos01.atlas.cogentco.com (154.24.6.237)  14.891 ms  16
.941 ms  16.702 ms
11  cogent.bos.ma.towerstream.com (38.104.186.82)  14.699 ms  14.188 ms  14.220 ms
12  g6-2.cr.bos1.ma.towerstream.com (64.119.143.81)  14.904 ms  14.903 ms  14.888 ms
13  69.38.149.18 (69.38.149.18)  18.293 ms  34.857 ms  33.138 ms
14  64.119.137.154 (64.119.137.154)  33.122 ms  36.814 ms  36.329 ms
15  world.pts.com (192.74.137.15)  34.369 ms  34.567 ms  29.696 ms

The mtr command differs from traceroute in several ways. First, like top, it provides a table of values that refreshes itself every second or so, allowing you to see how the values are updated over time. You can slow this down by giving the command a -i or --interval argument and specify the seconds you want to pass between each update.

It also shows you the packet loss -- like ping.

The mtr command also shows you a number of statistics for each leg in the route. The columns in the output (see example below) represent:

  • Snt -- number of packets sent
  • Loss% -- percentage of packets lost at each hop (can change with --report-cycles=# where # is replaced by the number of packets you want to send
  • Last -- latency of the last packet sent
  • Avg -- average latency
  • Best -- shortest round trip
  • Wrst -- longest round trip
  • StDev -- standard deviation

Last, Avg, Best, and Wrst are all provided in milliseconds

                                My traceroute  [v0.71]
boson.xyz.org (0.0.0.0)                                     Sun Aug 31 16
:22:55 2014
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                     Packets               Pings
 Host                                         Loss%  Last   Avg  Best  Wrst StDev
 1. 192.168.0.1                               50.0%   0.4   0.4   0.4   0.4   0.0
 2. ???
 3. gig1-6.umcp-core.net.doz.org                0.0%   1.6   4.0   1.6   6.6   2.5
 4. ten2-0.stpaul-core.net.doz.org              0.0%   2.7   2.8   2.7   2.8   0.1
 5. te4-3.ccr01.bwi01.atlas.cogentco.com        0.0%  91.5  32.3   2.7  91.5  51.2
 6. te4-2.ccr01.phl01.atlas.cogentco.com        0.0%   5.6  11.6   5.6  23.4  10.2
 7. te0-0-0-19.mpd21.jfk02.atlas.cogentco.com   0.0%   8.8   8.7   8.7   8.8   0.1
 8. be2095.ccr21.bos01.atlas.cogentco.com       0.0%  14.5  14.4  14.3  14.5   0.1
 9. te4-1.mag01.bos01.atlas.cogentco.com        0.0%  14.1  14.2  14.1  14.3   0.1
10. gi0-0-0-0.nr11.b000502-0.bos01.atlas.com    0.0%  14.7  14.6  14.6  14.7   0.1
11. cogent.bos.ma.towerstream.com               0.0%  14.1  14.1  14.1  14.1   0.1
12. g6-2.cr.bos1.ma.towerstream.com             0.0%  14.8  14.8  14.8  14.8   0.0
13. 69.38.149.18                                0.0%  24.0  26.9  24.0  29.7   4.1
14. 64.119.137.154                              0.0%  28.5  28.5  28.5  28.5   0.0
15. world.pts.com                               0.0%  23.3  22.2  21.1  23.3   1.5

Another often used mtr command is to use the -r or --report command. This gives you a static report (rather than updating every second). Instead, it runs through 10 iterations (or whatever you tell it using the -c (count) or --report-cycles option and shows you the result at the end.

You can ask for a report using syntax like this:

mtr --report <destination>
mtr <destination> --report
$ mtr world.pts.com --report
boson.xyz.org                    Snt: 10    Loss%  Last   Avg  Best  Wrst StDev
pix                                          50.0%   0.4   0.4   0.4   0.4   0.0
???                                          100.0   0.0   0.0   0.0   0.0   0.0
gig1-6.umcp-core.net.doz.org                  0.0%  12.6   4.0   1.5  12.6   3.6
ten2-0.stpaul-core.net.doz.org                0.0%   8.2   5.3   2.7  13.0   4.1
te4-3.ccr01.bwi01.atlas.cogentco.com          0.0%   2.9  33.4   2.6 139.2  52.1
te4-2.ccr01.phl01.atlas.cogentco.com          0.0%   5.7  52.9   5.5 201.2  74.9
te0-0-0-19.mpd21.jfk02.atlas.cogentco.com     0.0%   8.5   8.6   8.5   8.7   0.1
be2095.ccr21.bos01.atlas.cogentco.com         0.0%  14.4  14.6  14.3  15.0   0.2
te4-1.mag01.bos01.atlas.cogentco.com          0.0%  14.5  28.5  14.0 157.2  45.2
gi0-0-0-0.nr11.b000502-0.bos01.atlas.cogentc  0.0%  15.0  14.8  14.7  15.1   0.2
cogent.bos.ma.towerstream.com                 0.0%  14.1  27.0  14.0 136.0  38.4
g6-2.cr.bos1.ma.towerstream.com               0.0%  15.9  15.0  14.8  15.9   0.3
69.38.149.18                                  0.0%  22.6  23.5  18.3  34.2   4.7
64.119.137.154                               10.0%  23.0  25.4  19.6  32.0   4.7
world.pts.com                                 0.0%  21.9  23.7  19.2  29.9   3.6

Both packet loss and latency tell you a lot about the quality of your connections. A large loss will indicate a problem with the particular router. Note in the second line above, we see 100% loss. This router is not sending anything back to us, though this doesn't mean that it isn't a functional router. Obviously, the connections are reaching the final destination. But the router is probably not allowing icmp traffic to go back to the source or is taking too long. The ??? indicates timeouts. Some loss that you see may be due to rate limiting settings on routers.

Some people who use mtr routinely for troubleshooting network connections suggest that you run reports in both directions if you want to fully diagnose your connection issues.

What’s wrong? The new clean desk test
Join the discussion
Be the first to comment on this article. Our Commenting Policies