|
|
 | | From: | Redistributed | | Subject: | JMLR: Learning Hidden Variable Networks: The Information Bottleneck Approach | | Date: | Thu, 20 Jan 2005 18:55:03 GMT |
|
|
 | [[Redistributed from JMLR announce]]
~From: elm@cs.umass.edu
~Date: Thu, 13 Jan 2005 09:29:36 -0500
~Subject: [Jmlr-announce] Learning Hidden Variable Networks: The Information Bottleneck Approach
The Journal of Machine Learning Research (www.jmlr.org) is pleased to
announce publication of a new paper:
------------------------------------------------------------------------
-------
Learning Hidden Variable Networks: The Information Bottleneck Approach
Gal Elidan and Nir Friedman
JMLR 6 (Jan): 81--127, 2005
Abstract
A central challenge in learning probabilistic graphical models is
dealing with domains that involve hidden variables. The common approach
for learning model parameters in such domains is the expectation
maximization (EM) algorithm. This algorithm, however, can easily get
trapped in suboptimal local maxima. Learning the model structure is
even more challenging. The structural EM algorithm can adapt the
structure in the presence of hidden variables, but usually performs
poorly without prior knowledge about the cardinality and location of
the hidden variables. In this work, we present a general approach for
learning Bayesian networks with hidden variables that overcomes these
problems. The approach builds on the information bottleneck framework
of Tishby et al. (1999). We start by proving formal correspondence
between the information bottleneck objective and the standard
parametric EM functional. We then use this correspondence to construct
a learning algorithm that combines an information-theoretic smoothing
term with a continuation procedure. Intuitively, the algorithm bypasses
local maxima and achieves superior solutions by following a continuous
path from a solution of, an easy and smooth, target function, to a
solution of the desired likelihood function. As we show, our
algorithmic framework allows learning of the parameters as well as the
structure of a network. In addition, it also allows us to introduce new
hidden variables during model selection and learn their cardinality. We
demonstrate the performance of our procedure on several challenging
real-life data sets.
------------------------------------------------------------------------
------
This paper and previous papers are available electronically at
http://www.jmlr.org in PDF format. The papers of Volumes 1-4 were also
published in hardcopy by MIT Press; please see
http://mitpress.mit.edu/JMLR for details. Volume 5 and subsequent
volumes will be printed in hardcopy by Microtome Publishing. Please see
http://www.mtome.com/Publications/jmlr.html for details and ordering
information.
-Erik G. Learned-Miller
elm@cs.umass.edu
_________________
Erik G. Learned-Miller
University of Massachusetts Amherst
http://www.cs.umass.edu/~elm
(413) 545-2993
_______________________________________________
Jmlr-announce mailing list
Jmlr-announce@lists.csail.mit.edu
http://lists.csail.mit.edu/mailman/listinfo/jmlr-announce
[ comp.ai is moderated. To submit, just post and be patient, or if ]
[ that fails mail your article to , and ]
[ ask your news administrator to fix the problems with your system. ]
|
|
|
