Better Generative Models for Sequential Data Problems: Bidirectional Recurrent Mixture Density Networks

Abstract:
This paper describes bidirectional recurrent mixture density networks, which can model multi-modal distributions of the type   P( x _t | y )  and   P( x _t | x ₁ , x ₂ , . . ., x _{t -1} , y ), without any explicit assumptions about the use of context. These expressions occur frequently in pattern recognition problems with sequential data, for example in speech recognition. Experiments show that the proposed generative models give a higher likelihood on test data compared to a traditional modeling approach, indicating that they can summarize the statistical properties of the data better.}