Logistic Markov decision processes

Martin Mladenov, Ofer Meshi, Craig Boutilier, Gal Elidan, Dale Schuurmans, Tyler Lu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations


User modeling in advertising and recommendation has typically focused on myopic predictors of user responses. In this work, we consider the long-term decision problem associated with user interaction. We propose a concise specification of long-term interaction dynamics by combining factored dynamic Bayesian networks with logistic predictors of user responses, allowing state-of-the-art prediction models to be seamlessly extended. We show how to solve such models at scale by providing a constraint generation approach for approximate linear programming that overcomes the variable coupling and nonlinearity induced by the logistic regression predictor. The efficacy of the approach is demonstrated on advertising domains with up to 254 states and 239 actions.

Original languageAmerican English
Title of host publication26th International Joint Conference on Artificial Intelligence, IJCAI 2017
EditorsCarles Sierra
PublisherInternational Joint Conferences on Artificial Intelligence
Number of pages8
ISBN (Electronic)9780999241103
StatePublished - 2017
Externally publishedYes
Event26th International Joint Conference on Artificial Intelligence, IJCAI 2017 - Melbourne, Australia
Duration: 19 Aug 201725 Aug 2017

Publication series

NameIJCAI International Joint Conference on Artificial Intelligence
ISSN (Print)1045-0823


Conference26th International Joint Conference on Artificial Intelligence, IJCAI 2017


Dive into the research topics of 'Logistic Markov decision processes'. Together they form a unique fingerprint.

Cite this