The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
Este artigo propõe um método de retreinamento ajustável em camadas para computação em memória (CiM) FeFET de borda para compensar a degradação da precisão da rede neural (NN) por erros do dispositivo FeFET. O retreinamento proposto pode ajustar o número de camadas a serem retreinadas para reduzir a degradação da precisão da inferência por erros que ocorrem após o retreinamento. Os pesos do modelo NN original, treinados com precisão no data center em nuvem, são gravados no Edge FeFET CiM. Os pesos escritos são alterados por erros do dispositivo FeFET em campo. Ao retreinar parcialmente o modelo NN escrito, o método proposto combina as camadas afetadas por erros do modelo NN com as camadas retreinadas. A precisão da inferência é assim recuperada. Após o retreinamento, as camadas retreinadas são reescritas no CiM e afetadas novamente por erros do dispositivo. Na avaliação, inicialmente, é analisada a capacidade de recuperação do modelo NN por retreinamento parcial. Em seguida, a precisão da inferência após a reescrita é avaliada. A capacidade de recuperação é avaliada com erros típicos de memória não volátil (NVM): distribuição normal, deslocamento uniforme e inversão de bits. Para todos os tipos de erros, mais de 50% da porcentagem degradada de precisão de inferência é recuperada pelo retreinamento apenas da camada final totalmente conectada (FC) do Resnet-32. Para simular CiM de multiplicação local e acumulação global (LM-GA) FeFET, a capacidade de recuperação também é avaliada com erros FeFET modelados com base em medições FeFET. O retreinamento apenas da camada FC atinge uma taxa de recuperação de até 53%, 66% e 72% para variação de gravação FeFET, perturbação de leitura e retenção de dados, respectivamente. Além disso, apenas adicionar mais duas camadas de retreinamento melhora a taxa de recuperação em 20-30%. Para ajustar o número de camadas de retreinamento, a precisão da inferência após a reescrita é avaliada simulando os erros que ocorrem após o retreinamento. Quando erros típicos de NVM são injetados, é ideal treinar novamente a camada FC e 3-6 camadas de convolução do Resnet-32. O número ideal de camadas pode ser aumentado ou diminuído dependendo do equilíbrio entre o tamanho dos erros antes do retreinamento e dos erros após o retreinamento.
Shinsei YOSHIKIYO
The University of Tokyo
Naoko MISAWA
The University of Tokyo
Kasidit TOPRASERTPONG
The University of Tokyo
Shinichi TAKAGI
The University of Tokyo
Chihiro MATSUI
The University of Tokyo
Ken TAKEUCHI
The University of Tokyo
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copiar
Shinsei YOSHIKIYO, Naoko MISAWA, Kasidit TOPRASERTPONG, Shinichi TAKAGI, Chihiro MATSUI, Ken TAKEUCHI, "Write Variation & Reliability Error Compensation by Layer-Wise Tunable Retraining of Edge FeFET LM-GA CiM" in IEICE TRANSACTIONS on Electronics,
vol. E106-C, no. 7, pp. 352-364, July 2023, doi: 10.1587/transele.2022CDP0004.
Abstract: This paper proposes a layer-wise tunable retraining method for edge FeFET Computation-in-Memory (CiM) to compensate the accuracy degradation of neural network (NN) by FeFET device errors. The proposed retraining can tune the number of layers to be retrained to reduce inference accuracy degradation by errors that occur after retraining. Weights of the original NN model, accurately trained in cloud data center, are written into edge FeFET CiM. The written weights are changed by FeFET device errors in the field. By partially retraining the written NN model, the proposed method combines the error-affected layers of NN model with the retrained layers. The inference accuracy is thus recovered. After retraining, the retrained layers are re-written to CiM and affected by device errors again. In the evaluation, at first, the recovery capability of NN model by partial retraining is analyzed. Then the inference accuracy after re-writing is evaluated. Recovery capability is evaluated with non-volatile memory (NVM) typical errors: normal distribution, uniform shift, and bit-inversion. For all types of errors, more than 50% of the degraded percentage of inference accuracy is recovered by retraining only the final fully-connected (FC) layer of Resnet-32. To simulate FeFET Local-Multiply and Global-accumulate (LM-GA) CiM, recovery capability is also evaluated with FeFET errors modeled based on FeFET measurements. Retraining only FC layer achieves recovery rate of up to 53%, 66%, and 72% for FeFET write variation, read-disturb, and data-retention, respectively. In addition, just adding two more retraining layers improves recovery rate by 20-30%. In order to tune the number of retraining layers, inference accuracy after re-writing is evaluated by simulating the errors that occur after retraining. When NVM typical errors are injected, it is optimal to retrain FC layer and 3-6 convolution layers of Resnet-32. The optimal number of layers can be increased or decreased depending on the balance between the size of errors before retraining and errors after retraining.
URL: https://global.ieice.org/en_transactions/electronics/10.1587/transele.2022CDP0004/_p
Copiar
@ARTICLE{e106-c_7_352,
author={Shinsei YOSHIKIYO, Naoko MISAWA, Kasidit TOPRASERTPONG, Shinichi TAKAGI, Chihiro MATSUI, Ken TAKEUCHI, },
journal={IEICE TRANSACTIONS on Electronics},
title={Write Variation & Reliability Error Compensation by Layer-Wise Tunable Retraining of Edge FeFET LM-GA CiM},
year={2023},
volume={E106-C},
number={7},
pages={352-364},
abstract={This paper proposes a layer-wise tunable retraining method for edge FeFET Computation-in-Memory (CiM) to compensate the accuracy degradation of neural network (NN) by FeFET device errors. The proposed retraining can tune the number of layers to be retrained to reduce inference accuracy degradation by errors that occur after retraining. Weights of the original NN model, accurately trained in cloud data center, are written into edge FeFET CiM. The written weights are changed by FeFET device errors in the field. By partially retraining the written NN model, the proposed method combines the error-affected layers of NN model with the retrained layers. The inference accuracy is thus recovered. After retraining, the retrained layers are re-written to CiM and affected by device errors again. In the evaluation, at first, the recovery capability of NN model by partial retraining is analyzed. Then the inference accuracy after re-writing is evaluated. Recovery capability is evaluated with non-volatile memory (NVM) typical errors: normal distribution, uniform shift, and bit-inversion. For all types of errors, more than 50% of the degraded percentage of inference accuracy is recovered by retraining only the final fully-connected (FC) layer of Resnet-32. To simulate FeFET Local-Multiply and Global-accumulate (LM-GA) CiM, recovery capability is also evaluated with FeFET errors modeled based on FeFET measurements. Retraining only FC layer achieves recovery rate of up to 53%, 66%, and 72% for FeFET write variation, read-disturb, and data-retention, respectively. In addition, just adding two more retraining layers improves recovery rate by 20-30%. In order to tune the number of retraining layers, inference accuracy after re-writing is evaluated by simulating the errors that occur after retraining. When NVM typical errors are injected, it is optimal to retrain FC layer and 3-6 convolution layers of Resnet-32. The optimal number of layers can be increased or decreased depending on the balance between the size of errors before retraining and errors after retraining.},
keywords={},
doi={10.1587/transele.2022CDP0004},
ISSN={1745-1353},
month={July},}
Copiar
TY - JOUR
TI - Write Variation & Reliability Error Compensation by Layer-Wise Tunable Retraining of Edge FeFET LM-GA CiM
T2 - IEICE TRANSACTIONS on Electronics
SP - 352
EP - 364
AU - Shinsei YOSHIKIYO
AU - Naoko MISAWA
AU - Kasidit TOPRASERTPONG
AU - Shinichi TAKAGI
AU - Chihiro MATSUI
AU - Ken TAKEUCHI
PY - 2023
DO - 10.1587/transele.2022CDP0004
JO - IEICE TRANSACTIONS on Electronics
SN - 1745-1353
VL - E106-C
IS - 7
JA - IEICE TRANSACTIONS on Electronics
Y1 - July 2023
AB - This paper proposes a layer-wise tunable retraining method for edge FeFET Computation-in-Memory (CiM) to compensate the accuracy degradation of neural network (NN) by FeFET device errors. The proposed retraining can tune the number of layers to be retrained to reduce inference accuracy degradation by errors that occur after retraining. Weights of the original NN model, accurately trained in cloud data center, are written into edge FeFET CiM. The written weights are changed by FeFET device errors in the field. By partially retraining the written NN model, the proposed method combines the error-affected layers of NN model with the retrained layers. The inference accuracy is thus recovered. After retraining, the retrained layers are re-written to CiM and affected by device errors again. In the evaluation, at first, the recovery capability of NN model by partial retraining is analyzed. Then the inference accuracy after re-writing is evaluated. Recovery capability is evaluated with non-volatile memory (NVM) typical errors: normal distribution, uniform shift, and bit-inversion. For all types of errors, more than 50% of the degraded percentage of inference accuracy is recovered by retraining only the final fully-connected (FC) layer of Resnet-32. To simulate FeFET Local-Multiply and Global-accumulate (LM-GA) CiM, recovery capability is also evaluated with FeFET errors modeled based on FeFET measurements. Retraining only FC layer achieves recovery rate of up to 53%, 66%, and 72% for FeFET write variation, read-disturb, and data-retention, respectively. In addition, just adding two more retraining layers improves recovery rate by 20-30%. In order to tune the number of retraining layers, inference accuracy after re-writing is evaluated by simulating the errors that occur after retraining. When NVM typical errors are injected, it is optimal to retrain FC layer and 3-6 convolution layers of Resnet-32. The optimal number of layers can be increased or decreased depending on the balance between the size of errors before retraining and errors after retraining.
ER -