The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
Este artigo aborda o problema de seleção de um subconjunto significativo de recursos candidatos para uso na regressão linear múltipla. Bertsimas et ai. [5] propuseram recentemente o algoritmo discreto de primeira ordem (DFO) para encontrar com eficiência soluções quase ótimas para este problema. No entanto, este algoritmo é incapaz de escapar de soluções localmente ótimas. Para resolver isso, propomos um algoritmo estocástico discreto de primeira ordem (SDFO) para seleção de subconjuntos de recursos. Neste algoritmo, perturbações aleatórias são adicionadas a uma sequência de soluções candidatas como um meio de escapar de soluções localmente ótimas, o que amplia a gama de soluções detectáveis. Além disso, derivamos o tamanho ideal do passo na direção gradiente-descida para acelerar a convergência do algoritmo. Também fazemos uso eficaz do L2-termo de regularização para melhorar o desempenho preditivo de um modelo de regressão de subconjunto resultante. Os resultados da simulação demonstram que nosso algoritmo supera substancialmente o algoritmo DFO original. Nosso algoritmo foi superior em desempenho preditivo ao laço e à seleção passo a passo também.
Kota KUDO
University of Tsukuba
Yuichi TAKANO
University of Tsukuba
Ryo NOMURA
Waseda University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copiar
Kota KUDO, Yuichi TAKANO, Ryo NOMURA, "Stochastic Discrete First-Order Algorithm for Feature Subset Selection" in IEICE TRANSACTIONS on Information,
vol. E103-D, no. 7, pp. 1693-1702, July 2020, doi: 10.1587/transinf.2019EDP7274.
Abstract: This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2019EDP7274/_p
Copiar
@ARTICLE{e103-d_7_1693,
author={Kota KUDO, Yuichi TAKANO, Ryo NOMURA, },
journal={IEICE TRANSACTIONS on Information},
title={Stochastic Discrete First-Order Algorithm for Feature Subset Selection},
year={2020},
volume={E103-D},
number={7},
pages={1693-1702},
abstract={This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.},
keywords={},
doi={10.1587/transinf.2019EDP7274},
ISSN={1745-1361},
month={July},}
Copiar
TY - JOUR
TI - Stochastic Discrete First-Order Algorithm for Feature Subset Selection
T2 - IEICE TRANSACTIONS on Information
SP - 1693
EP - 1702
AU - Kota KUDO
AU - Yuichi TAKANO
AU - Ryo NOMURA
PY - 2020
DO - 10.1587/transinf.2019EDP7274
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E103-D
IS - 7
JA - IEICE TRANSACTIONS on Information
Y1 - July 2020
AB - This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.
ER -