Representation, Learning, and Reasoning (RLR) Lab

Our group’s research interests are in the area of automated reasoning, planning and reinforcement learning, key building blocks of artificial intelligence. Our goal is to develop effective decision making strategies for intelligent agents that help agents to act in a coordinated manner to achiever their goal.

We use knowledge representation frameworks such as graphical models (Bayesian networks, Markov random fields) to represent and exploit the structure present in real world planning problems. We use a data driven approach to learn the model of the environment. Using learned models, we construct feature rich domain simulators that are used within a planning and reinforcement learning framework to reason about the best course of actions agents must take to perform their tasks.

Some of the keywords that can be associated with our research include graphical models, probabilistic inference, sequential decision making, multiagent systems, Markov decision processes, mathematical optimization and machine learning. Some ongoing projects are listed below.

alt text 

Large Scale Multiagent Deep Reinforcement Learning

Our increasingly interconnected urban environments provide several opportunities to deploy intelligent agents——from self-driving cars, ships to aerial drones——that promise to radically improve productivity and safety. Achieving coordination among agents in such urban settings presents several algorithmic challenges—ability to scale to thousands of agents, addressing uncertainty, and partial observability in the environment. Furthermore, in such complex transportation systems, only a simulator for the domain is available rather than a complete model.

My group is actively developing reinforcement learning (RL) based strategies as well as feature-rich traffic simulators (for autonomous vessels, drones) that enable coordination in such large multiagent systems with thousands of agents. E.g., the (left) figure shows a snapshot from our maritime traffic simulator. Accompanying paper develops deep RL based strategies to coordinate vessel traffic to increase safety of navigation by reducing traffic congestion, and also takes into account realistic domains constraints such as vessels cannot fully stop midwater, and movement of vessels is uncertain affected by several factors (such as weather).

Relevant publications along this direction are:

alt text 

Multiagent Planning and Decision Making Using Probabilistic Inference

Traditionally, the area of decision making and planning in agent-based models such as Markov decision processes (MDPs) and its multiagent extension (Dec-MDPs) has evolved relatively independently from the area of machine learning. Recently some intriguing connections have been developed between problems in machine learning (ML) such as likelihood maximization and that of planning and decision making in agent-based models. My ongoing work explores this connection in the area of multiagent systems leading to efficient and scalable algorithms for several problems. Applying ML techniques to that of decision making promises to be a very exciting research direction for the future.

Relevant publications along this direction are:

alt text 

Decision Making and Learning for Computational Sustainability

Computational sustainability is an emerging interdisciplinary field that aims to bring together computational techniques from different areas of computer science and operations research for intelligent decision making to manage and conserve society and environment’s limited resources. Addressing important problems in sustainability such as network design for spatial conservation planning, understanding the population dynamics of an endangered species, and decision making for smart power grid management represent some of the core issues of my research agenda.

Relevant publications along this direction are:

alt text 

Approximate Inference in Graphical Models

Probabilistic graphical models (PGMs) are a powerful framework that arise in a variety of practical applications such as multiagent systems, computer vision and bioinformatics. PGMs succinctly encode the structure present in real world problems by combining the language of graphs with probability theory. Such structure can then be exploited to develop efficient inference algorithms.

In my work, I have developed several message-passing based inference algorithm for the problem of maximum-a-posteriori (MAP) estimation in PGMs. I have been also active at addressing inference problems for collective graphical models (CGMs) that are graphical models defined over aggregate-level data rather than the individual model. CGMs are particularly useful when one has access to aggregate level data such as modeling bird migration, visitor diffusion in theme parks and learning from anonymized datasets.

Relevant publications along this direction are:

Students

I am fortunate to work with an excellent group of students! Current students/research engineers are:

Students/research engineers I have worked with in the past include:

  • Thien Nguyen-Duc, Ph.D. (2014 - 2018) (Co-advised with HC Lau). Last known employment: Research Scientist@IBM Research Singapore

  • Tarun Gupta (Master’s thesis co-advisor with Praveen Paruchuri, research engineer July 2018 - Jun 2019). Last known employment: Grab Singapore

  • Saumya Bhatnagar, Masters in IS (2017), research engineer (2017 - 2018). Last known employment: UNiCEN, Singapore

  • Lucas Agussurja, research engineer (2016-2018). Last known employment: NUS