PCM: Precision-Controlled Memory System for Energy Efficient Deep Neural Network Training

Boyeal Kim, Sang Hyun Lee, Hyun Kim, Duy Thanh Nguyen, Minh Son Le, Ik Joon Chang, Dohun Kwon, Jin Hyeok Yoo, Jun Won Choi, Hyuk Jae Lee

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

Deep neural network (DNN) training suffers from the significant energy consumption in memory system, and most existing energy reduction techniques for memory system have focused on introducing low precision that is compatible with computing unit (e.g., FP16, FP8). These researches have shown that even in learning the networks with FP16 data precision, it is possible to provide training accuracy as good as FP32, de facto standard of the DNN training. However, our extensive experiments show that we can further reduce the data precision while maintaining the training accuracy of DNNs, which can be obtained by truncating some least significant bits (LSBs) of FP16, named as hard approximation. Nevertheless, the existing hard-ware structures for DNN training cannot efficiently support such low precision. In this work, we propose a novel memory system architecture for GPUs, named as precision-controlled memory system (PCM), which allows for flexible management at the level of hard approximation. PCM provides high DRAM bandwidth by distributing each precision to different channels with as transposed data mapping on DRAM. In addition, PCM supports fine-grained hard approximation in the L1 data cache using software-controlled registers, which can reduce data movement and thereby improve energy saving and system performance. Furthermore, PCM facilitates the reduction of data maintenance energy, which accounts for a considerable portion of memory energy consumption, by controlling refresh period of DRAM. The experimental results show that in training CIFAR-100 dataset on Resnet-20 with precision tuning, PCM achieves energy saving and performance enhancement by 66% and 20%, respectively, without loss of accuracy.

Original languageEnglish
Title of host publicationProceedings of the 2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020
EditorsGiorgio Di Natale, Cristiana Bolchini, Elena-Ioana Vatajelu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1199-1204
Number of pages6
ISBN (Electronic)9783981926347
DOIs
StatePublished - 2020 Mar
Event2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020 - Grenoble, France
Duration: 2020 Mar 92020 Mar 13

Publication series

NameProceedings of the 2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020

Conference

Conference2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020
CountryFrance
CityGrenoble
Period20/03/920/03/13

Keywords

  • Approximate Computing
  • Deep Neural Network
  • General Purpose Graphic Processing Unit
  • High Bandwidth Memory
  • Precision Control
  • Refresh Period Control

Fingerprint Dive into the research topics of 'PCM: Precision-Controlled Memory System for Energy Efficient Deep Neural Network Training'. Together they form a unique fingerprint.

Cite this