An efficient algorithm for measuring medium- to large-sized flows in network traffic

Ashwin Lall, Mitsunori Ogihara, Jun Xu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

It has been well recognized that identifying very large flows (i.e., elephants) in a network traffic stream is important for a variety of network applications ranging from traffic engineering to anomaly detection. However, we found that many of these applications have an increasing need to monitor not only the few largest flows (say top 20), but also all of the medium-sized flows (say top 20,000). Unfortunately, existing techniques for identifying elephant flows at high link speeds are not suitable and cannot be trivially extended for identifying the medium-sized flows. In this work, we propose a hybrid SRAM/DRAM algorithm for monitoring all elephant and medium-sized flows with strong accuracy guarantees. We employ a synopsis data structure (sketch) in SRAM to filter out small flows and preferentially sample medium and large flows to a flow table in DRAM. Our key contribution is to show how to maximize the use of SRAM and DRAM available to us by using a SRAM/DRAM hybrid data structure that can achieve more than an order of magnitude higher SRAM efficiency than previous methods. We design a quantization scheme that allows our algorithm to "read just enough" from the sketch at SRAM speed, without sacrificing much estimation accuracy. We provide analytical guarantees on the accuracy of the estimation and validate these by means of trace-driven evaluation using real-world packet traces.

Original languageEnglish (US)
Title of host publicationIEEE INFOCOM 2009 - The 28th Conference on Computer Communications
Pages2711-2715
Number of pages5
DOIs
StatePublished - Oct 12 2009
Event28th Conference on Computer Communications, IEEE INFOCOM 2009 - Rio de Janeiro, Brazil
Duration: Apr 19 2009Apr 25 2009

Publication series

NameProceedings - IEEE INFOCOM
ISSN (Print)0743-166X

Other

Other28th Conference on Computer Communications, IEEE INFOCOM 2009
CountryBrazil
CityRio de Janeiro
Period4/19/094/25/09

    Fingerprint

ASJC Scopus subject areas

  • Computer Science(all)
  • Electrical and Electronic Engineering

Cite this

Lall, A., Ogihara, M., & Xu, J. (2009). An efficient algorithm for measuring medium- to large-sized flows in network traffic. In IEEE INFOCOM 2009 - The 28th Conference on Computer Communications (pp. 2711-2715). [5062217] (Proceedings - IEEE INFOCOM). https://doi.org/10.1109/INFCOM.2009.5062217