How do you visualize your traffic flow (NetFlow) logs?

November 18, 2008, 9:05 AM — In my last blog post I showed how you collect NetFlow records and format them suitable for visualization. The next step, after the data has been processed and prepared in a CSV format, is to pick a graph or visual representation that is suitable for the problem at hand. Relationships between data entries are best visualized with link graphs. GraphViz is a tool that can be used to generate these link graphs. Unfortunately, the input format for GraphViz is a DOT file, which is a fairly complicated representation of a link graph. In order to translate CSV files to DOT files, I am going to use a tool called AfterGlow.
AfterGlow reads CSV data. Each edge in the graph is represented by a single CSV entry. For example, a node from 10.0.0.1 to 10.0.0.2 is generate by the following entry:

10.0.0.1,10.0.0.2

To generate the DOT file from our CSV input, run the CSV data through the following command:

cat file.csv | afterglow.pl -t > graph.dot

The DOT file now needs to be visualized with GraphViz. There are various layout algorithms that GraphViz supports, each launched through its own command. An example graph is generated as follows:

cat graph.dot | neato -Tgif -o graph.gif

To try out the other layouts, exchange neato in the above command with circo, dot, fdp, twopi.

If you want to get a bit more sophisticated with formatting the graph, you can use a graph property file and pass it to AfterGlow. The property file is used to assign color to nodes, filter nodes, or cluster them. A sample property file looks as follows:

color="yellow" if (field()=~/^192\.168\..*/);
color="greenyellow" if (field()=~/^10\..*/);
color="lightyellow4" if (field()=~/^172\.16\..*/);
color="red"

This sample property file assigns colors to the nodes based on their subnet. Nodes that belong to private address spaces are drawn in yellow. Other nodes are drawn in red. To tell AfterGlow to use this property file, use the following command:

cat file.csv | afterglow.pl -t -c sample.properties | neato -Tgif -o graph.gif

Have you generated your own link graphs? Share your examples and experiences on SecViz.

Hope to see you at my visualization workshop on Thursday the 20th of November in Hong Kong at the IS Summit 2008.