The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
Hadoop, uma estrutura de processamento distribuído para big data, é agora amplamente utilizada para processamento multimídia. No entanto, ao processar dados de vídeo de um sistema de arquivos distribuído Hadoop (HDFS), é gerado tráfego de rede desnecessário devido a uma política ineficiente de fatia de bloco HDFS para quadros de imagem em arquivos de vídeo. Propomos uma nova política de replicação de blocos para resolver este problema e comparar o HDFS recentemente proposto com o HDFS original através de extensos experimentos. O HDFS proposto reduz o tráfego de rede e aumenta a localidade entre núcleos de processamento e locais de arquivos.
Cheolgi KIM
Korea Aerospace University
Daechul LEE
Korea Aerospace University
Jaehyun LEE
Korea Aerospace University
Jaehwan LEE
Korea Aerospace University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copiar
Cheolgi KIM, Daechul LEE, Jaehyun LEE, Jaehwan LEE, "An Efficient Block Assignment Policy in Hadoop Distributed File System for Multimedia Data Processing" in IEICE TRANSACTIONS on Information,
vol. E102-D, no. 8, pp. 1569-1571, August 2019, doi: 10.1587/transinf.2019EDL8016.
Abstract: Hadoop, a distributed processing framework for big-data, is now widely used for multimedia processing. However, when processing video data from a Hadoop distributed file system (HDFS), unnecessary network traffic is generated due to an inefficient HDFS block slice policy for picture frames in video files. We propose a new block replication policy to solve this problem and compare the newly proposed HDFS with the original HDFS via extensive experiments. The proposed HDFS reduces network traffic, and increases locality between processing cores and file locations.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2019EDL8016/_p
Copiar
@ARTICLE{e102-d_8_1569,
author={Cheolgi KIM, Daechul LEE, Jaehyun LEE, Jaehwan LEE, },
journal={IEICE TRANSACTIONS on Information},
title={An Efficient Block Assignment Policy in Hadoop Distributed File System for Multimedia Data Processing},
year={2019},
volume={E102-D},
number={8},
pages={1569-1571},
abstract={Hadoop, a distributed processing framework for big-data, is now widely used for multimedia processing. However, when processing video data from a Hadoop distributed file system (HDFS), unnecessary network traffic is generated due to an inefficient HDFS block slice policy for picture frames in video files. We propose a new block replication policy to solve this problem and compare the newly proposed HDFS with the original HDFS via extensive experiments. The proposed HDFS reduces network traffic, and increases locality between processing cores and file locations.},
keywords={},
doi={10.1587/transinf.2019EDL8016},
ISSN={1745-1361},
month={August},}
Copiar
TY - JOUR
TI - An Efficient Block Assignment Policy in Hadoop Distributed File System for Multimedia Data Processing
T2 - IEICE TRANSACTIONS on Information
SP - 1569
EP - 1571
AU - Cheolgi KIM
AU - Daechul LEE
AU - Jaehyun LEE
AU - Jaehwan LEE
PY - 2019
DO - 10.1587/transinf.2019EDL8016
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E102-D
IS - 8
JA - IEICE TRANSACTIONS on Information
Y1 - August 2019
AB - Hadoop, a distributed processing framework for big-data, is now widely used for multimedia processing. However, when processing video data from a Hadoop distributed file system (HDFS), unnecessary network traffic is generated due to an inefficient HDFS block slice policy for picture frames in video files. We propose a new block replication policy to solve this problem and compare the newly proposed HDFS with the original HDFS via extensive experiments. The proposed HDFS reduces network traffic, and increases locality between processing cores and file locations.
ER -