Krill: A Compiler and Runtime System for Concurrent Graph Processing
Event Type
Paper
Machine Learning and Artificial Intelligence
Numerical Algorithms
TP
TimeWednesday, 17 November 202111:30am - 12pm CST
Location230-231-232
DescriptionAs a large number of emerging graph applications spread across different domains, the need for processing massive concurrent graph jobs (CGJs) is increasing. However, existing graph processing systems designed for a single job cannot efficiently tackle multiple CGJs, where they suffer from interfering memory access patterns and inefficient property management. In this paper, we introduce Krill, a compiler and runtime system for processing concurrent graph jobs. In the compiler, we propose leveraging the property buffer to easily write and manage property data. In the runtime system, we propose a novel technique named graph kernel fusion to reduce memory accesses, which fuses all the jobs and processes them as a whole. Experimental results show our system significantly reduces the number of memory accesses for CGJs by more than 6x compared with the baseline, and achieves up to 7.67x speedup with 3.84x shorter response latency compared with GraphM, the state-of-the-art graph system.
Download PDF
Archive view
