5G/6G时代的安全分布式机器学习

By Huawei Huang, Dec. 31, 2019

      随着AI芯片的制造成本逐渐下降,越来越多的移动设备逐渐具备机器学习的能力。同时,网络连接质量作为分布式机器学习的瓶颈,将会随着5G/6G时代的到来大大地得到提升。为了应对即将到来的5G/6G时代新型分布式机器学习的需求,我们需要一个能支持大规模的分布式机器学习的框架。尤其重要的是,在大规模用户参与的情况下,分布式机器学习的安全问题应该引起足够的重视。

图 1. The proposed PIRATE framework has two critical components: 1) reliability assessment; 2) blockchain (BC) systems for distributed SGD (D-SGD). Gradient aggregations and model parameters are protected by blockchains. Meanwhile, reliability assessment determines the participants of distributed learning tasks.
图 2. While adopting the “Ring Allreduce” mechanism, malicious attackers can perform attacking from both inside and outside. (1) Attackers from the outside can contaminate training models in target nodes. (2) The outside attackers can also attack partial gradient-aggregation. (3) Byzantine computing nodes can send harmful aggregations that damage the convergence of learning tasks.

      理论上来讲,恶意攻击者可以攻击分布式机器学习的各个环节。针对妨碍训练收敛的任意攻击行为,我们提出了图1所示的 PIRATE:一个基于区块链的安全分布式学习框架。由于区块链技术验证的灵活性,此框架可不限于保护本文中所述的训练收敛性,而是在其他安全保护方面也具有巨大的潜能。基于本框架,更多的保护机制可以被开发出来,如针对参与分布式学习的设备进行隐私保护,针对 Model Poisoning Attack 的保护,为所有参与者提供激励机制等。

      如图1所示,PIRATE主要由两部分组成,一是设备可靠性分析,用以分析设备的可靠性,进而决定设备能否能参与学习任务;二是基于多个分片链的安全SGD(Stochastic Gradient Descent)框架。

      我们采取了去中心化的 Ring AllReduce 结构(图2)。这种架构可以更好地分担网络压力,并且可以在验证计算结果的同时进行梯度计算。同时,为了让节点在 Ring AllReduce 结构下高效、可验证地沟通,我们运用了基于分片的区块链。其中,我们把所有节点分成多个委员会,每个节点只需要验证自己委员会内的梯度计算。这种分布式验证极大地降低了广播所带来的时延。

       经过模拟实验,在5G网络条件、较大训练模型、以及大规模用户参与的条件下,PIRATE 比同类框架LearningChain更节省存储空间,时间上更加高效。

       该项研究的前期工作已经上传到 arXiv。论文一作为本实验室研究生0年级学生周思聪同学,第一篇论文写得很有前瞻性,可圈可点。

       为了阅读方便,论文的 arXiv 链接为:https://arxiv.org/abs/1912.07860

       更多信息请参照该论文的 【researchGate 页面】,欢迎下载并留言。

===============================================
【English Version】

With the production cost of AI chips gradually reduced to an acceptable level, mobile devices are better equipped with computational resources for machine learning. Meanwhile, as the bottleneck of distributed machine learning, network condition would be substantially improved as we march into the 5G era. To exploit the merits of 5G/6G networks, a large-scale distributed learning framework is in need. Particularly, in the large-scale scenario, security problems become even more critical.

  To protect against arbitrary convergence hindrance attacks, we propose PIRATE, a blockchain-based secure distributed learning framework. The framework has great potential utilizing the verification flexibility of blockchain techniques. Such flexibility enables more protection mechanisms to be built on top of the framework, e.g., privacy protection, Model Poisoning Attack protection, incentive mechanism, etc.

  As shown in Figure 1, PIRATE has two components: 1) reliability assessment, which decides whether a device could take part in a learning task; 2) a secure SGD framework based on multiple shard chains.

  We utilize the decentralized architecture, Ring AllReduce (Figure 2), which can better leverage network resources, and enables devices to verify computation results while computing gradients.

  Furthermore, in order to conduct efficient and verifiable communication under the Ring AllReduce setting, we utilize a sharding-based blockchain technique. In particular, we divide nodes into multiple committees, in which nodes are only required to verify gradients within their committee. Such division greatly reduces the latency of broadcasting.

  Simulation experiments show that, under the condition of 5G/6G networks, relatively large training models, and large-scale participants, PIRATE outperforms a similar framework, LearningChain, in terms of storage complexity and latency.

  Please feel free to download and read from the following URLs:

  arXiv: https://arxiv.org/abs/1912.07860

  ResearchGate: PDF
————————————
作者:黄华威

发表评论

电子邮件地址不会被公开。 必填项已用*标注