Neural Gas for Dictionary Learning
Robust and Fast Learning of Sparse Codes with Stochastic Gradient Descent
Particular classes of signals, as for example natural images, can be encoded sparsely if appropriate dictionaries are used. Finding such dictionaries based on data samples, however, is a difficult optimization task. We have shown that simple stochastic gradient descent, besides being much faster, leads to superior dictionaries compared to other state-of-the-art methods ,. Moreover, we introduced the so-called Bag of Pursuits methodas an extension of Orthogonal Matching Pursuit,  and have shown that it provides an improved approximation of the optimal sparse coefficients and, therefore, significantly improves the performance of dictionary learning approaches that employ it in order to determine the coefficients within the learning process . Finally, we have shown how the Bag of Pursuits and a generalized version of the Neural Gas algorithm can be used to derive an even more powerful method for soft-competitive sparse coding .
We provide a an OCTAVE/MATLAB package that contains an optimized implementation of Neural Gas for Dictionary Learning and the Bag of Pursuits method for sparse approximation:
(1) Download the Neural Gas for Dictionary Learning (NGDL) package (Warning! This file is about 150MB since it also contains all the training data that is required in order to perform the included example experiments).
(2) Extract the tar archive: tar -xzf NGDL.tar.gz
The archive contains precompiled MATLAB binaries for the linux x86-64 architecture. There are also OCTAVE binaries for linux x86-64 architecture included. For other platforms/operating systems, you have to compile the sources:
(3) Add the -msse2 compiler option to the CFLAGS variable of the MATLAB MEX compiler (mexopt.sh)
(4) Enter the NGDL directory
(5) Compile the C sources by typing make. Note that the MATLAB MEX compiler has to be in the path.
Note 1: If you don't want to use SSE2 you can disable it by removing the __USE_SSE__ compiler option in the make file.
Note 2: It is possible to compile the package for octave usage. In order to obtain octave compatible mex files modify the CC variable in the make file (replace mex by mkoctfile, see make file for more information).
Note 3: On x86-64 architecture use the __USE_SSE64__ compiler option (see make file).
Please contact email@example.com if you have further questions.
- Kai Labusch, Erhardt Barth, and Thomas Martinetz. Robust and Fast Learning of Sparse Codes with Stochastic Gradient Descent. IEEE Transactions on Selected Topics in Signal Processing, 5(5):1048-1060, 2011.
- Kai Labusch, Erhardt Barth, and Thomas Martinetz. Soft-competitive Learning of Sparse Codes and its Application to Image Reconstruction. Neurocomputing, 74(9):1418-1428, 2011.
- Kai Labusch, Erhardt Barth, and Thomas Martinetz. Bag of Pursuits and Neural Gas for Improved Sparse Coding. In Gilbert Saporta, editor, Proceedings of the 19th International Conference on Computational Statistics, pages 327-336. Springer, 2010.
- Kai Labusch and Thomas Martinetz. Learning Sparse Codes for Image Reconstruction. In Michel Verleysen, editor, Proceedings of the 18th European Symposium on Artificial Neural Networks, pages 241-246. D-Side Publishers, 2010.