Distributed adaptive networks provide a scalable, fault-tolerant, and decentralized platform for distributed learning. Effective learning in such networks requires a structured approach based on a parallel computing model to ensure smooth interaction between parallel software and hardware. While the bulk synchronous parallel (BSP) model can be applied, in heterogeneous environments, it suffers from the straggler problem caused by global barrier synchronization, increasing node idle time. This paper introduces FBSP (Federated Bulk Synchronous Parallel), an enhanced BSP model designed for heterogeneous adaptive networks. FBSP uses content–structural clustering to group nodes into regions and employs a two-level barrier synchronization—regional and global—to optimize computational flow, thereby reducing communication overhead and idle time.

Keywords