Report of the Second Workshop on the Usage of NetFlow/IPFIX in Network Management

More information about the current work in the IETF can be found in the IPFIX status pages [15].

Schmoll proposed to encrypt exported IPFIX data and to decrypt them only when strictly necessary. His solution uses a different encryption key for each collector device, allowing exporting devices to decide which collectors will decrypt which portion of the data. All communication for key exchange is protected by standard TLS (Transport Layer Security), and all standard security measures–such as protection by firewalls and access control policies–should also be in place. However, a comprehensive evaluation of the effectiveness of his approach is still to be performed.

Cristian Morariu’s presentation targeted the bottlenecks in handling NetFlow/IPFIX data. Since NetFlow/IPFIX meters are often used in high-speed network environments, the infrastructure to transport and to process those data must be designed to support heavy workloads. Bottlenecks can occur if, for example, NetFlow/IPFIX data arrive at a collector device at rates higher than the writing speed of the storage hardware, if the bandwidth available in the network is not sufficient, or if the time required to process a NetFlow/IPFIX record is longer than the inter-arrival time of such records. These bottlenecks are normally addressed at the metering point, by sampling packets or flows before exporting any data. However, some applications require highly accurate measurements, and sampling approaches may have a negative impact on that.

Morariu proposed a new architecture, suitable for situations in which sampling is not acceptable. His solution, based on the Kademlia distributed hash table [11], aims to increase the number of flows that can be processed, by distributing the workload across several network nodes. Furthermore, his solution is more robust, since peer-to-peer networks provide redundancy and avoid single points of failure. Although a prototypical implementation already exists, further analysis is needed to ensure its feasibility.

In the second session, new applications for NetFlow/IPFIX were discussed. The session was opened by Nikolay Melnikov (Jacobs University Bremen), who is researching methods for the identification of users, on the basis of network flow analysis. Each user has his or her own individual browsing style that leads to specific patterns of flow data. Melnikov’s goal is to identify users automatically, comparing unknown NetFlow/IPFIX records to known examples of users’ flow signatures. Even though this research is still in its early stages, first results are promising: using cross-correlation as proximity measure, the duration distribution of flows from four volunteer users could be differentiated. The next steps of his research will include the creation of more features from original data and the selection of the most discriminative features. However, all those tasks depend on the availability of data from new volunteers.

Tim Kleefass (SWITCH/University of Stuttgart) presented a new application for NetFlow/IPFIX data that can help network operators to identify remote connectivity problems. Network operators need to know about problems before their customers notice them, even when those problems are not directly related to their services or products. As an example, all traffic going to YouTube was lost for several minutes in 2008, due to a wrong configuration created by a telecommunication company in Pakistan. Users all around the world were affected, but without knowing exactly who caused the problem. Kleefass proposed to compare the number of flows passing edge routers within a network to and from a remote location. Using those numbers, he updates a connectivity matrix every 5 min. In the event of remote connectivity problems, those numbers will get unbalanced, indicating an abnormal situation. Some applications that have unbalanced traffic in normal situations, such as Skype, must be filtered out to avoid false alarms. Using logs from the Swiss Research and Educational Network (SWITCHlan), Kleefass showed that this approach successfully detects outages at remote Internet locations.

Jochen Kögel (Universtität Stuttgart) closed this session, explaining how NetFlow/IPFIX data can be used to extract performance metrics in enterprise networks. As described in RFC3917 [14] and RFC5472 [17], quality of service monitoring is one of the target applications of IPFIX, but a comprehensive research into what can be extracted from IPFIX data is still lacking. Kögel presented methods to extract RTT (Round Trip Time) and one-way delay figures from flow data. The RTT can be extracted from a single router if all traffic (in both directions) is routed through it. In that case, the RTT will be the difference between the starting times of a pair of related flows (request/response flows). Delay estimation, on the other hand, requires information from several routers on the path between the end nodes. The delay can be calculated based on the starting time of the same flow, but collected at different points (assuming that all devices have synchronized clocks). Experiments showed good results when comparing his calculations with active measurements.

The remaining presentations focused on NetFlow/IPFIX usage in practice, and in particular on (1) real-world experiences of a NetFlow tool-vendor, (2) a visualization tool for NetFlow data, and (3) an analysis of the bandwidth behavior of network flows.

Most of IsarNet’s customers are still using NetFlow version 5. However, the demand for version 9 is increasing. Bourges argued that Flexible NetFlow has several promising features, but version 5 is enough to meet the requirements of most customers. Moreover, the deployment of Flexible NetFlow faces a dilemma: tool vendors require that customers’ hardware/software infrastructure supports this new version before they start providing products for it. However, customers are not willing to upgrade their infrastructure if there is no application that supports Flexible NetFlow.

Rick Hofstede (University of Twente) proposed a novel way to visualize flow data, based on geographic information. He implemented a plug-in for NfSen [12] that interfaces with IP2Location [8] and the Google Maps API [7]. The result is a web application that displays geographic information about the network traffic. His tool provides zoom levels that allow discrimination between various aspects of network information, while retaining an intuitive interface.

The study was based on data collected at the University of Twente during 2007. At this stage of his research, only those flows with more than 100 MB were considered. Sadre concluded that most flows in his data set have a constant throughput, but there are some large deviations. Furthermore, more precise throughput estimation is achieved as the observing time of a flow gets longer.