Applied Mathematics/Learning Theory

Theoretical Prediction Accuracy of Nonnegative Matrix Factorization

Introduction

Non-negative matrix factorization ( NMF ) is a method of decomposing a matrix with nonnegative value entries into products of two non-negative value elements. Signal processing such as image and sound, bioinformatics, text mining, consumer analysis, etc. are widely used as composite data analysis technology. It is used as a knowledge discovery method in various fields numerically and experimentally. For example, it is applied to decomposition of power spectrum in signal processing, analysis of purchasing factors in consumer analysis and extraction of purchasing patterns such as representative products in the factor. Despite its wide applicability, its mathematical structure, in particular the estimation accuracy, has not been clarified much. If the theoretical value of the estimation precision is known, a criterion can be obtained as to whether or not the numerical experiment is successful. Conversely, if there is no theoretical value, the correctness of the result obtained by numerical calculation is not guaranteed at all.

In general Bayesian Learning, there is the theoretical method to clarify expected generalization error. Moreover, if the model is singular i.e. posterior cannot be approximated to normal distribution, Bayesian method is superior to maximum likelihood method and maximum a posteriori method in that the generalization error is smaller. That is why we study theoretical prediction accuracy of Bayesian NMF.

Main Theorem

Please see this preprint: ''Upper Bound of Bayesian Generalization Error in Non-Negative Matrix Factorization''.

Numerical Experiments

Under Construction

Model Selection by singular Bayesian Information Criterion (sBIC)

Under Construction