The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
Com o desenvolvimento de câmeras e sensores e a disseminação da computação em nuvem, os registros de vida podem ser facilmente adquiridos e armazenados em residências em geral para os diversos serviços que utilizam os registros. No entanto, é difícil analisar imagens em movimento adquiridas por sensores domésticos em tempo real usando aprendizado de máquina porque o tamanho dos dados é muito grande e a complexidade computacional é muito alta. Além disso, coletar e acumular na nuvem imagens em movimento capturadas em casa e que podem ser utilizadas para identificar indivíduos pode invadir a privacidade dos usuários do aplicativo. Propomos um método de processamento distribuído na borda e na nuvem que aborda a latência do processamento e as questões de privacidade. No lado da borda (sensor), extraímos vetores de características de pontos-chave humanos de imagens em movimento usando OpenPose, que é uma biblioteca de estimativa de pose. No lado da nuvem, reconhecemos ações por aprendizado de máquina usando apenas os vetores de recursos. Neste estudo, comparamos as precisões de reconhecimento de ação de vários métodos de aprendizado de máquina. Além disso, medimos o tempo de processamento da análise no sensor e na nuvem para investigar a viabilidade de reconhecimento de ações em tempo real. Em seguida, avaliamos o sistema proposto comparando-o com o modelo 3D ResNet em experimentos de reconhecimento. Os resultados experimentais demonstram que a precisão do reconhecimento de ações é maior quando se usa LSTM e que a introdução do abandono no reconhecimento de ações usando 100 categorias alivia o overfitting porque os modelos podem aprender ações humanas mais genéricas, aumentando a variedade de ações. Além disso, foi demonstrado que o pré-processamento usando OpenPose no lado do sensor pode reduzir substancialmente a quantidade de transferência do sensor para a nuvem.
Chikako TAKASAKI
Ochanomizu University
Atsuko TAKEFUSA
National Institute of Informatics
Hidemoto NAKADA
National Institute of Advanced Industrial Science and Technology (AIST)
Masato OGUCHI
Ochanomizu University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copiar
Chikako TAKASAKI, Atsuko TAKEFUSA, Hidemoto NAKADA, Masato OGUCHI, "Action Recognition Using Pose Data in a Distributed Environment over the Edge and Cloud" in IEICE TRANSACTIONS on Information,
vol. E104-D, no. 5, pp. 539-550, May 2021, doi: 10.1587/transinf.2020DAP0009.
Abstract: With the development of cameras and sensors and the spread of cloud computing, life logs can be easily acquired and stored in general households for the various services that utilize the logs. However, it is difficult to analyze moving images that are acquired by home sensors in real time using machine learning because the data size is too large and the computational complexity is too high. Moreover, collecting and accumulating in the cloud moving images that are captured at home and can be used to identify individuals may invade the privacy of application users. We propose a method of distributed processing over the edge and cloud that addresses the processing latency and the privacy concerns. On the edge (sensor) side, we extract feature vectors of human key points from moving images using OpenPose, which is a pose estimation library. On the cloud side, we recognize actions by machine learning using only the feature vectors. In this study, we compare the action recognition accuracies of multiple machine learning methods. In addition, we measure the analysis processing time at the sensor and the cloud to investigate the feasibility of recognizing actions in real time. Then, we evaluate the proposed system by comparing it with the 3D ResNet model in recognition experiments. The experimental results demonstrate that the action recognition accuracy is the highest when using LSTM and that the introduction of dropout in action recognition using 100 categories alleviates overfitting because the models can learn more generic human actions by increasing the variety of actions. In addition, it is demonstrated that preprocessing using OpenPose on the sensor side can substantially reduce the transfer quantity from the sensor to the cloud.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2020DAP0009/_p
Copiar
@ARTICLE{e104-d_5_539,
author={Chikako TAKASAKI, Atsuko TAKEFUSA, Hidemoto NAKADA, Masato OGUCHI, },
journal={IEICE TRANSACTIONS on Information},
title={Action Recognition Using Pose Data in a Distributed Environment over the Edge and Cloud},
year={2021},
volume={E104-D},
number={5},
pages={539-550},
abstract={With the development of cameras and sensors and the spread of cloud computing, life logs can be easily acquired and stored in general households for the various services that utilize the logs. However, it is difficult to analyze moving images that are acquired by home sensors in real time using machine learning because the data size is too large and the computational complexity is too high. Moreover, collecting and accumulating in the cloud moving images that are captured at home and can be used to identify individuals may invade the privacy of application users. We propose a method of distributed processing over the edge and cloud that addresses the processing latency and the privacy concerns. On the edge (sensor) side, we extract feature vectors of human key points from moving images using OpenPose, which is a pose estimation library. On the cloud side, we recognize actions by machine learning using only the feature vectors. In this study, we compare the action recognition accuracies of multiple machine learning methods. In addition, we measure the analysis processing time at the sensor and the cloud to investigate the feasibility of recognizing actions in real time. Then, we evaluate the proposed system by comparing it with the 3D ResNet model in recognition experiments. The experimental results demonstrate that the action recognition accuracy is the highest when using LSTM and that the introduction of dropout in action recognition using 100 categories alleviates overfitting because the models can learn more generic human actions by increasing the variety of actions. In addition, it is demonstrated that preprocessing using OpenPose on the sensor side can substantially reduce the transfer quantity from the sensor to the cloud.},
keywords={},
doi={10.1587/transinf.2020DAP0009},
ISSN={1745-1361},
month={May},}
Copiar
TY - JOUR
TI - Action Recognition Using Pose Data in a Distributed Environment over the Edge and Cloud
T2 - IEICE TRANSACTIONS on Information
SP - 539
EP - 550
AU - Chikako TAKASAKI
AU - Atsuko TAKEFUSA
AU - Hidemoto NAKADA
AU - Masato OGUCHI
PY - 2021
DO - 10.1587/transinf.2020DAP0009
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E104-D
IS - 5
JA - IEICE TRANSACTIONS on Information
Y1 - May 2021
AB - With the development of cameras and sensors and the spread of cloud computing, life logs can be easily acquired and stored in general households for the various services that utilize the logs. However, it is difficult to analyze moving images that are acquired by home sensors in real time using machine learning because the data size is too large and the computational complexity is too high. Moreover, collecting and accumulating in the cloud moving images that are captured at home and can be used to identify individuals may invade the privacy of application users. We propose a method of distributed processing over the edge and cloud that addresses the processing latency and the privacy concerns. On the edge (sensor) side, we extract feature vectors of human key points from moving images using OpenPose, which is a pose estimation library. On the cloud side, we recognize actions by machine learning using only the feature vectors. In this study, we compare the action recognition accuracies of multiple machine learning methods. In addition, we measure the analysis processing time at the sensor and the cloud to investigate the feasibility of recognizing actions in real time. Then, we evaluate the proposed system by comparing it with the 3D ResNet model in recognition experiments. The experimental results demonstrate that the action recognition accuracy is the highest when using LSTM and that the introduction of dropout in action recognition using 100 categories alleviates overfitting because the models can learn more generic human actions by increasing the variety of actions. In addition, it is demonstrated that preprocessing using OpenPose on the sensor side can substantially reduce the transfer quantity from the sensor to the cloud.
ER -