Occasionally we work with interesting network bugs or we want to learn more about how our service behaves outside of the application layer. Two tools that help me in those occasions are tshark and tcpdump.
Installation
Let’s assume we work on a unix system. Then to install tcpdump we simple execute:
|
1 |
$ apt install tcpdump |
Installing tshark is simple too:
|
1 |
$ apt install libcap2-bin tshark |
Example API
As a demonstration, we are going to utilize a clean and small Sinatra server. For this example a working Ruby installation is required. See the official docs on how to install Ruby properly on a local machine: https://www.ruby-lang.org/en/documentation/installation/
Going back to our example, first, create the repo:
|
1 |
$ mkdir booklist && cd booklist |
Install sinatra:
|
1 |
$ gem install sinatra |
Create the example API:
|
1 2 3 4 5 6 7 8 |
$ cat server.rb require 'sinatra' require 'json' get '/' do ["pragmatic programmer", "clean code"].to_json end |
Fire up the server:
|
1 2 3 4 5 6 7 8 9 |
$ ruby server.rb == Sinatra (v2.0.4) has taken the stage on 4567 for development with backup from Puma Puma starting in single mode... * Version 3.12.0 (ruby 2.4.0-p0), codename: Llamas in Pajamas * Min threads: 0, max threads: 16 * Environment: development * Listening on tcp://localhost:4567 Use Ctrl-C to stop |
It works:
|
1 2 3 4 5 6 |
$ curl -X GET localhost:4567 | jq . [ "pragmatic programmer", "clean code" ] |
tcpdump
tcpdump is available on most unix systems, so we can use it on a small remote sever where tshark would be most probably an overkill. It provides decoding so we can investigate how our services interact with the network.
We will work with the lo interface for this example:
|
1 |
$ tcpdump -D |
Execute:
|
1 |
$ sudo tcpdump -i lo -A |
Hit the service:
|
1 |
$ curl -X GET localhost:4567 |
Now notice that tcpdump has captured the traffic:
|
1 2 3 4 5 6 7 8 |
HTTP/1.1 200 OK Content-Type: text/html;charset=utf-8 X-XSS-Protection: 1; mode=block X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN Content-Length: 37 ["pragmatic programmer","clean code"] |
tshark
tshark is a powerful sniffer with many filters which can decode traffic and provides tools for running more complex analysis on it.
tshark can behave exactly like tcpdump:
|
1 |
$ tshark -i lo --color |
Depending on the problem, the best solution may be to combine the powers of these tools. A usual case is to create a file with captured decoded traffic with tcpdump and then run analysis on the file with tshark. Or just do both with tshark.
Let’s explore how to do this on the previous example.
Capture the traffic to a packet capture(pcap) file:
|
1 2 |
$ touch dump $ tshark -i lo -w dump.pcap |
Analyze it:
|
1 |
$ tshark -r dump.pcap |
For example, HTTP analysis:
|
1 |
$ tshark -r dump.pcap -Y http.request -T fields -e http.host -e http.user_agent | sort | uniq -c | sort -n |
We could also use wireshark for the last step which provides a nice GUI.
Outro
Learning more about these tools has helped me analyze and research solutions on more complicated problems, which in turn help me grow as an engineer and problem solver. Some good references for learning more about these tools are:
1) tutorial: https://danielmiessler.com/study/tcpdump/
2) how tcp works: https://medium.com/@eranda/analyze-tcp-dumps-a089c2644f19
3) book: https://www.goodreads.com/book/show/505564.The_TCP_IP_Guide