Scaling Up Heterogeneous Waveform Clustering for Long-Duration Monitoring Signal Acquisition, Analysis, and Interaction: Bridging Big Data Analytics with Measurement Instrument Usage Pattern

Masaharu Goto, Naoki Kobayashi, Gang Ren, Mitsunori Ogihara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Modern oscilloscopes, digitizers and data loggers generate a large amount of waveform data for long-duration waveform capturing and analysis. The contrast of time scales of long-duration waveform capturing (e.g., hours or days in high sampling rate) and analysis (e.g., signal fragments of several microseconds) produces unique big data challenges. The proposed long-duration waveform clustering algorithms are designed for signal waveform analysis and user interaction for various 'big-data' waveform analysis scenarios. To cope with the real-time processing demand and the hardware constraints of the target platforms, the proposed algorithm utilizes multiple layers of data pre-sorting, database query, and waveform similarity-based clustering for versatile speed-precision tradeoffs. We integrated the system as an intuitive big waveform data analytics framework which provides unprecedented performance and productivity to engineers and scientists. Experimental result shows superb speed and data volume capability.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE International Conference on Big Data, Big Data 2019
EditorsChaitanya Baru, Jun Huan, Latifur Khan, Xiaohua Tony Hu, Ronay Ak, Yuanyuan Tian, Roger Barga, Carlo Zaniolo, Kisung Lee, Yanfang Fanny Ye
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1794-1803
Number of pages10
ISBN (Electronic)9781728108582
DOIs
StatePublished - Dec 2019
Event2019 IEEE International Conference on Big Data, Big Data 2019 - Los Angeles, United States
Duration: Dec 9 2019Dec 12 2019

Publication series

NameProceedings - 2019 IEEE International Conference on Big Data, Big Data 2019

Conference

Conference2019 IEEE International Conference on Big Data, Big Data 2019
CountryUnited States
CityLos Angeles
Period12/9/1912/12/19

Keywords

  • Clustering
  • Long-duration waveform
  • Measurement instruments
  • Real-time signal processing
  • Waveform analysis

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Information Systems
  • Information Systems and Management

Fingerprint Dive into the research topics of 'Scaling Up Heterogeneous Waveform Clustering for Long-Duration Monitoring Signal Acquisition, Analysis, and Interaction: Bridging Big Data Analytics with Measurement Instrument Usage Pattern'. Together they form a unique fingerprint.

  • Cite this

    Goto, M., Kobayashi, N., Ren, G., & Ogihara, M. (2019). Scaling Up Heterogeneous Waveform Clustering for Long-Duration Monitoring Signal Acquisition, Analysis, and Interaction: Bridging Big Data Analytics with Measurement Instrument Usage Pattern. In C. Baru, J. Huan, L. Khan, X. T. Hu, R. Ak, Y. Tian, R. Barga, C. Zaniolo, K. Lee, & Y. F. Ye (Eds.), Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019 (pp. 1794-1803). [9006208] (Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData47090.2019.9006208