Abstract
It has been well recognized that identifying very large flows (i.e., elephants) in a network traffic stream is important for a variety of network applications ranging from traffic engineering to anomaly detection. However, we found that many of these applications have an increasing need to monitor not only the few largest flows (say top 20), but also all of the medium-sized flows (say top 20,000). Unfortunately, existing techniques for identifying elephant flows at high link speeds are not suitable and cannot be trivially extended for identifying the medium-sized flows. In this work, we propose a hybrid SRAM/DRAM algorithm for monitoring all elephant and medium-sized flows with strong accuracy guarantees. We employ a synopsis data structure (sketch) in SRAM to filter out small flows and preferentially sample medium and large flows to a flow table in DRAM. Our key contribution is to show how to maximize the use of SRAM and DRAM available to us by using a SRAM/DRAM hybrid data structure that can achieve more than an order of magnitude higher SRAM efficiency than previous methods. We design a quantization scheme that allows our algorithm to "read just enough" from the sketch at SRAM speed, without sacrificing much estimation accuracy. We provide analytical guarantees on the accuracy of the estimation and validate these by means of trace-driven evaluation using real-world packet traces.
| Original language | English (US) |
|---|---|
| Title of host publication | Proceedings - IEEE INFOCOM |
| Pages | 2711-2715 |
| Number of pages | 5 |
| DOIs | |
| State | Published - 2009 |
| Event | 28th Conference on Computer Communications, IEEE INFOCOM 2009 - Rio de Janeiro, Brazil Duration: Apr 19 2009 → Apr 25 2009 |
Other
| Other | 28th Conference on Computer Communications, IEEE INFOCOM 2009 |
|---|---|
| Country | Brazil |
| City | Rio de Janeiro |
| Period | 4/19/09 → 4/25/09 |
Fingerprint
ASJC Scopus subject areas
- Computer Science(all)
- Electrical and Electronic Engineering
Cite this
An efficient algorithm for measuring medium- to large-sized flows in network traffic. / Lall, Ashwin; Ogihara, Mitsunori; Xu, Jun.
Proceedings - IEEE INFOCOM. 2009. p. 2711-2715 5062217.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - An efficient algorithm for measuring medium- to large-sized flows in network traffic
AU - Lall, Ashwin
AU - Ogihara, Mitsunori
AU - Xu, Jun
PY - 2009
Y1 - 2009
N2 - It has been well recognized that identifying very large flows (i.e., elephants) in a network traffic stream is important for a variety of network applications ranging from traffic engineering to anomaly detection. However, we found that many of these applications have an increasing need to monitor not only the few largest flows (say top 20), but also all of the medium-sized flows (say top 20,000). Unfortunately, existing techniques for identifying elephant flows at high link speeds are not suitable and cannot be trivially extended for identifying the medium-sized flows. In this work, we propose a hybrid SRAM/DRAM algorithm for monitoring all elephant and medium-sized flows with strong accuracy guarantees. We employ a synopsis data structure (sketch) in SRAM to filter out small flows and preferentially sample medium and large flows to a flow table in DRAM. Our key contribution is to show how to maximize the use of SRAM and DRAM available to us by using a SRAM/DRAM hybrid data structure that can achieve more than an order of magnitude higher SRAM efficiency than previous methods. We design a quantization scheme that allows our algorithm to "read just enough" from the sketch at SRAM speed, without sacrificing much estimation accuracy. We provide analytical guarantees on the accuracy of the estimation and validate these by means of trace-driven evaluation using real-world packet traces.
AB - It has been well recognized that identifying very large flows (i.e., elephants) in a network traffic stream is important for a variety of network applications ranging from traffic engineering to anomaly detection. However, we found that many of these applications have an increasing need to monitor not only the few largest flows (say top 20), but also all of the medium-sized flows (say top 20,000). Unfortunately, existing techniques for identifying elephant flows at high link speeds are not suitable and cannot be trivially extended for identifying the medium-sized flows. In this work, we propose a hybrid SRAM/DRAM algorithm for monitoring all elephant and medium-sized flows with strong accuracy guarantees. We employ a synopsis data structure (sketch) in SRAM to filter out small flows and preferentially sample medium and large flows to a flow table in DRAM. Our key contribution is to show how to maximize the use of SRAM and DRAM available to us by using a SRAM/DRAM hybrid data structure that can achieve more than an order of magnitude higher SRAM efficiency than previous methods. We design a quantization scheme that allows our algorithm to "read just enough" from the sketch at SRAM speed, without sacrificing much estimation accuracy. We provide analytical guarantees on the accuracy of the estimation and validate these by means of trace-driven evaluation using real-world packet traces.
UR - http://www.scopus.com/inward/record.url?scp=70349675311&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=70349675311&partnerID=8YFLogxK
U2 - 10.1109/INFCOM.2009.5062217
DO - 10.1109/INFCOM.2009.5062217
M3 - Conference contribution
AN - SCOPUS:70349675311
SN - 9781424435135
SP - 2711
EP - 2715
BT - Proceedings - IEEE INFOCOM
ER -