Neural Processing Letters, ( ISI ), Volume (46), No (1), Year (2017-8) , Pages (15-28)

Title : ( A New Sparse Learning Machine )

Authors: Mojtaba Nayyeri , Alaleh Maskooki , Reza Monsefi ,

Citation: BibTeX | EndNote

Abstract

Many algorithms have been proposed so far for pruning and sparse approximation of feedforward neural networks with random weights in order to obtain compact networks which are fast and robust on various datasets. One drawback of the randomization process is that the resulted weight vectors might be highly correlated. It has been shown that ensemble classifiers’ error depends on the amount of error correlation between them. Thus, decrease in correlation between output vectors must lead to generation of more efficient hidden nodes. In this research a new learning algorithm called New Sparse Learning Machine (NSLM) for single-hidden layer feedforward networks is proposed for regression and classification. In the first phase, the algorithm creates hidden layer with small correlation among nodes by orthogonalizing the columns of the output matrix. Then in the second phase, using L1-norm minimization problem, NSLM makes the components of the solution vector become zero as many as possible. The resulted network has higher degree of sparsity while the accuracy is maintained or improved. Therefore, the method leads to a new network with a better generalization performance. Numerical comparisons on several classification and regression datasets confirm the expected improvement in comparison to the basic network.

Keywords

, Feedforward neural networks , Sparse learning machine, Approximation algorithm , L1-norm minimization, Error correlation, Gram–Schmidt orthogonalization
برای دانلود از شناسه و رمز عبور پرتال پویا استفاده کنید.

@article{paperid:1061269,
author = {Nayyeri, Mojtaba and Maskooki, Alaleh and Monsefi, Reza},
title = {A New Sparse Learning Machine},
journal = {Neural Processing Letters},
year = {2017},
volume = {46},
number = {1},
month = {August},
issn = {1370-4621},
pages = {15--28},
numpages = {13},
keywords = {Feedforward neural networks ;Sparse learning machine; Approximation algorithm ; L1-norm minimization;Error correlation; Gram–Schmidt orthogonalization},
}

[Download]

%0 Journal Article
%T A New Sparse Learning Machine
%A Nayyeri, Mojtaba
%A Maskooki, Alaleh
%A Monsefi, Reza
%J Neural Processing Letters
%@ 1370-4621
%D 2017

[Download]