目录概MotivationZeROZeRO-OffloadZeRO-InfiniteZeRO++代码Rajbhandari S., Rasley J., Ruwase O. and He Y. ZeRO: Memory optimizations toward training trillion parameter models. InSC, 2020.Ren J., Rajbhandari S., Aminabadi R. Y., Ruwase O., Yang S., Zhang M., Li…