In this paper, we propose a general two-objective Markov Decision Process (MDP) modeling paradigm for automated negotiation with incomplete information, in which preference elicitation alternates with negotiation actions, with the objective to optimize negotiation outcomes. The key ingredient in our MDP framework is a stochastic utility model governed by a Gaussian law, formalizing the agent's belief (uncertainty) over the user's preferences. Our belief model is fairly general and can be updated in real time as new data becomes available, which makes it a fundamental modeling tool.

Representing Users in a Negotiation (RUN): An Autonomous Negotiator Under Preference Uncertainty
The 18th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS '19)
This work was funded by the The Netherlands Organisation for Scientific Research (NWO); grant id nwo/639.021.751 - Representing Users in a Negotiation (RUN): An Autonomous Negotiator Under Preference Uncertainty
Intelligent and autonomous systems

Leahu, H, Kaisers, M, & Baarslag, T. (2019). Preference Learning in Automated Negotiation Using Gaussian Uncertainty Models. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (pp. 2087–2089).