IEEE Transactions on Parallel and Distributed Systems, ( ISI ), Volume (31), No (4), Year (2020-4) , Pages (766-778)

Title : ( cCUDA: Effective Co-Scheduling of Concurrent Kernels on GPUs )

Authors: SeyedKazem Shekofteh , Hamid Noori , Mahmoud Naghibzadeh , Holger Froning , Hadi Sadoghi Yazdi ,

Access to full-text not allowed by authors

Citation: BibTeX | EndNote


While GPUs are meantime omnipresent for many scientific and technical computations, they still continue to evolve as processors. An important recent feature is the ability to execute multiple kernels concurrently via queue streams. However, experiments show that different parameters including the behavior of kernels, the order of kernel launches and other execution configurations, e.g., the number of concurrent thread blocks, may result in different execution time for concurrent kernel execution. Since kernels may have different resource requirements, they can be classified into different classes, which are traditionally assumed as either memory-bound or compute-bound. However, a kernel may belong to the different classes on different hardware according to the hardware resources. In this paper, the definition of kernel mix intensity is introduced. Based on this, a scheduling framework called concurrent CUDA (cCUDA) is proposed to co-schedule the concurrent kernels more efficiently. It first profiles and ranks kernels with different execution behaviors and then takes the kernel resource requirements into account to partition thread blocks of different kernels and overlap them to better utilize the GPU resources. Experimental results on real hardware demonstrate performance improvement in terms of execution time of up to 1.86x, and an average speedup of 1.28x for a wide range of kernels. cCUDA is available at


, Kernel, scheduling, concurrent kernel execution, stream, resource management
برای دانلود از شناسه و رمز عبور پرتال پویا استفاده کنید.

author = {Shekofteh, SeyedKazem and Noori, Hamid and Naghibzadeh, Mahmoud and Holger Froning and Sadoghi Yazdi, Hadi},
title = {cCUDA: Effective Co-Scheduling of Concurrent Kernels on GPUs},
journal = {IEEE Transactions on Parallel and Distributed Systems},
year = {2020},
volume = {31},
number = {4},
month = {April},
issn = {1045-9219},
pages = {766--778},
numpages = {12},
keywords = {Kernel; scheduling; concurrent kernel execution; stream; resource management},


%0 Journal Article
%T cCUDA: Effective Co-Scheduling of Concurrent Kernels on GPUs
%A Shekofteh, SeyedKazem
%A Noori, Hamid
%A Naghibzadeh, Mahmoud
%A Holger Froning
%A Sadoghi Yazdi, Hadi
%J IEEE Transactions on Parallel and Distributed Systems
%@ 1045-9219
%D 2020