TY - GEN
T1 - An efficient algorithm for measuring medium- to large-sized flows in network traffic
AU - Lall, Ashwin
AU - Ogihara, Mitsunori
AU - Xu, Jun
PY - 2009
Y1 - 2009
N2 - It has been well recognized that identifying very large flows (i.e., elephants) in a network traffic stream is important for a variety of network applications ranging from traffic engineering to anomaly detection. However, we found that many of these applications have an increasing need to monitor not only the few largest flows (say top 20), but also all of the medium-sized flows (say top 20,000). Unfortunately, existing techniques for identifying elephant flows at high link speeds are not suitable and cannot be trivially extended for identifying the medium-sized flows. In this work, we propose a hybrid SRAM/DRAM algorithm for monitoring all elephant and medium-sized flows with strong accuracy guarantees. We employ a synopsis data structure (sketch) in SRAM to filter out small flows and preferentially sample medium and large flows to a flow table in DRAM. Our key contribution is to show how to maximize the use of SRAM and DRAM available to us by using a SRAM/DRAM hybrid data structure that can achieve more than an order of magnitude higher SRAM efficiency than previous methods. We design a quantization scheme that allows our algorithm to "read just enough" from the sketch at SRAM speed, without sacrificing much estimation accuracy. We provide analytical guarantees on the accuracy of the estimation and validate these by means of trace-driven evaluation using real-world packet traces.
AB - It has been well recognized that identifying very large flows (i.e., elephants) in a network traffic stream is important for a variety of network applications ranging from traffic engineering to anomaly detection. However, we found that many of these applications have an increasing need to monitor not only the few largest flows (say top 20), but also all of the medium-sized flows (say top 20,000). Unfortunately, existing techniques for identifying elephant flows at high link speeds are not suitable and cannot be trivially extended for identifying the medium-sized flows. In this work, we propose a hybrid SRAM/DRAM algorithm for monitoring all elephant and medium-sized flows with strong accuracy guarantees. We employ a synopsis data structure (sketch) in SRAM to filter out small flows and preferentially sample medium and large flows to a flow table in DRAM. Our key contribution is to show how to maximize the use of SRAM and DRAM available to us by using a SRAM/DRAM hybrid data structure that can achieve more than an order of magnitude higher SRAM efficiency than previous methods. We design a quantization scheme that allows our algorithm to "read just enough" from the sketch at SRAM speed, without sacrificing much estimation accuracy. We provide analytical guarantees on the accuracy of the estimation and validate these by means of trace-driven evaluation using real-world packet traces.
UR - http://www.scopus.com/inward/record.url?scp=70349675311&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=70349675311&partnerID=8YFLogxK
U2 - 10.1109/INFCOM.2009.5062217
DO - 10.1109/INFCOM.2009.5062217
M3 - Conference contribution
AN - SCOPUS:70349675311
SN - 9781424435135
T3 - Proceedings - IEEE INFOCOM
SP - 2711
EP - 2715
BT - IEEE INFOCOM 2009 - The 28th Conference on Computer Communications
T2 - 28th Conference on Computer Communications, IEEE INFOCOM 2009
Y2 - 19 April 2009 through 25 April 2009
ER -