The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
Este estudo considerou uma extensão de um método de regularização esparsa com escalonamento, especialmente em métodos de limiarização que são exemplos simples e típicos de modelagem esparsa. Neste estudo, no cenário de um problema de regressão ortogonal não paramétrica, desenvolvemos e analisamos um método de limiarização no qual os estimadores de limiarização suave são expandidos independentemente por valores de escala empíricos. Os valores de escala têm um hiperparâmetro comum que é uma ordem de expansão de um valor de escala ideal para atingir um limite rígido. Nós simplesmente nos referimos a este estimador como um estimador de limiar suave escalonado. O método de limiar suave escalonado é um método de ponte entre os métodos de limiar suave e duro. Este novo estimador é de fato consistente com um estimador LASSO adaptativo no caso ortogonal; ou seja, é, portanto, outra derivação de um estimador LASSO adaptativo. É um método geral que inclui limiar suave e garrote não negativo como casos especiais. Posteriormente, derivamos o grau de liberdade do limiar suave escalonado no cálculo da estimativa de risco imparcial de Stein. Descobrimos que ele é decomposto no grau de liberdade da limiarização suave e no termo restante conectado à limiarização rígida. Como o grau de liberdade reflete o grau de sobreajuste, isso implica que o limiar suave escalonado tem outra fonte de sobreajuste além do número de componentes não removidos. O resultado teórico foi verificado por meio de um exemplo numérico simples. Neste processo, também focamos na não monotonicidade no termo restante do grau de liberdade acima e descobrimos que, em um cenário de amostra esparso e grande, ela é causada principalmente por componentes inúteis que não estão relacionados à função alvo.
Katsuyuki HAGIWARA
Mie University
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copiar
Katsuyuki HAGIWARA, "Bridging between Soft and Hard Thresholding by Scaling" in IEICE TRANSACTIONS on Information,
vol. E105-D, no. 9, pp. 1529-1536, September 2022, doi: 10.1587/transinf.2021EDP7223.
Abstract: This study considered an extension of a sparse regularization method with scaling, especially in thresholding methods that are simple and typical examples of sparse modeling. In this study, in the setting of a non-parametric orthogonal regression problem, we developed and analyzed a thresholding method in which soft thresholding estimators are independently expanded by empirical scaling values. The scaling values have a common hyper-parameter that is an order of expansion of an ideal scaling value to achieve hard thresholding. We simply refer to this estimator as a scaled soft thresholding estimator. The scaled soft thresholding method is a bridge method between soft and hard thresholding methods. This new estimator is indeed consistent with an adaptive LASSO estimator in the orthogonal case; i.e., it is thus an another derivation of an adaptive LASSO estimator. It is a general method that includes soft thresholding and non-negative garrote as special cases. We subsequently derived the degree of freedom of the scaled soft thresholding in calculating the Stein's unbiased risk estimate. We found that it is decomposed into the degree of freedom of soft thresholding and the remainder term connecting to the hard thresholding. As the degree of freedom reflects the degree of over-fitting, this implies that the scaled soft thresholding has an another source of over-fitting in addition to the number of un-removed components. The theoretical result was verified by a simple numerical example. In this process, we also focused on the non-monotonicity in the above remainder term of the degree of freedom and found that, in a sparse and large sample setting, it is mainly caused by useless components that are not related to the target function.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2021EDP7223/_p
Copiar
@ARTICLE{e105-d_9_1529,
author={Katsuyuki HAGIWARA, },
journal={IEICE TRANSACTIONS on Information},
title={Bridging between Soft and Hard Thresholding by Scaling},
year={2022},
volume={E105-D},
number={9},
pages={1529-1536},
abstract={This study considered an extension of a sparse regularization method with scaling, especially in thresholding methods that are simple and typical examples of sparse modeling. In this study, in the setting of a non-parametric orthogonal regression problem, we developed and analyzed a thresholding method in which soft thresholding estimators are independently expanded by empirical scaling values. The scaling values have a common hyper-parameter that is an order of expansion of an ideal scaling value to achieve hard thresholding. We simply refer to this estimator as a scaled soft thresholding estimator. The scaled soft thresholding method is a bridge method between soft and hard thresholding methods. This new estimator is indeed consistent with an adaptive LASSO estimator in the orthogonal case; i.e., it is thus an another derivation of an adaptive LASSO estimator. It is a general method that includes soft thresholding and non-negative garrote as special cases. We subsequently derived the degree of freedom of the scaled soft thresholding in calculating the Stein's unbiased risk estimate. We found that it is decomposed into the degree of freedom of soft thresholding and the remainder term connecting to the hard thresholding. As the degree of freedom reflects the degree of over-fitting, this implies that the scaled soft thresholding has an another source of over-fitting in addition to the number of un-removed components. The theoretical result was verified by a simple numerical example. In this process, we also focused on the non-monotonicity in the above remainder term of the degree of freedom and found that, in a sparse and large sample setting, it is mainly caused by useless components that are not related to the target function.},
keywords={},
doi={10.1587/transinf.2021EDP7223},
ISSN={1745-1361},
month={September},}
Copiar
TY - JOUR
TI - Bridging between Soft and Hard Thresholding by Scaling
T2 - IEICE TRANSACTIONS on Information
SP - 1529
EP - 1536
AU - Katsuyuki HAGIWARA
PY - 2022
DO - 10.1587/transinf.2021EDP7223
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E105-D
IS - 9
JA - IEICE TRANSACTIONS on Information
Y1 - September 2022
AB - This study considered an extension of a sparse regularization method with scaling, especially in thresholding methods that are simple and typical examples of sparse modeling. In this study, in the setting of a non-parametric orthogonal regression problem, we developed and analyzed a thresholding method in which soft thresholding estimators are independently expanded by empirical scaling values. The scaling values have a common hyper-parameter that is an order of expansion of an ideal scaling value to achieve hard thresholding. We simply refer to this estimator as a scaled soft thresholding estimator. The scaled soft thresholding method is a bridge method between soft and hard thresholding methods. This new estimator is indeed consistent with an adaptive LASSO estimator in the orthogonal case; i.e., it is thus an another derivation of an adaptive LASSO estimator. It is a general method that includes soft thresholding and non-negative garrote as special cases. We subsequently derived the degree of freedom of the scaled soft thresholding in calculating the Stein's unbiased risk estimate. We found that it is decomposed into the degree of freedom of soft thresholding and the remainder term connecting to the hard thresholding. As the degree of freedom reflects the degree of over-fitting, this implies that the scaled soft thresholding has an another source of over-fitting in addition to the number of un-removed components. The theoretical result was verified by a simple numerical example. In this process, we also focused on the non-monotonicity in the above remainder term of the degree of freedom and found that, in a sparse and large sample setting, it is mainly caused by useless components that are not related to the target function.
ER -