No Travel? No Problem.

Remote Participation
The Service Analysis and Network Diagnosis Data Pipeline
Event Type
Workshop
Tags
Big Data
Cloud and Distributed Computing
Data Analytics
Data Management
Datacenter
Networks
Performance
Quantum Computing
Registration Categories
W
TimeMonday, 15 November 202110:30am - 11am CST
Location222
DescriptionModern network performance monitoring toolkits, such as perfSONAR, take a remarkable number of measurements about the local network environment. To gain a complete picture of network performance, however, one needs to aggregate data across a large number of endpoints. The Service Analysis and Network Diagnosis (SAND) data pipeline collects data from diverse sources and ingests these measurements into a message bus. The message bus allows the project to send the data to multiple consumers, including a tape archive, an Elasticsearch database, and a peer infrastructure at CERN. In this paper, we explain the architecture and evolution of the SAND data pipeline, the scale of the resulting dataset, and how it supports a wide variety of network analysis applications.
Back To Top Button