Why SoftMax? Full Bayesian filtering of arbitrary states.
Preface
This document provides the rough, unformatted run-through of the paper being submitted to the Robotics Science and Systems 2015 Workshop on Model Learning for Human-Robot Communication.
Big Picture
Answers to Heilmeier's Catechism:
What are you trying to do? Articulate your objectives using absolutely no jargon.
We're developing a technique for humans to inform robots of the world around them -- to develop a model for a human sensor.
How is it done today, and what are the limits of current practice?
Most researchers focus on providing robots with a spatial understanding of the environment, but this doesn't allow for many of the descriptions in human language. For example, most spatial-semantic systems would simply prune "in front of the red door" to "in front of the door".
What's new in your approach and why do you think it will be successful?
Our approach is more general than simply focusing on Euclidian space: we want to translate arbitrary state space, which includes Euclidian space. A state could be position, velocity, colour, a target's tactics -- anything!
This approach will be successful because it captures the richness of human language instead of focusing solely on prepositional relationships, allowing for far more information to be provided by a human sensor.
Who cares?
Anyone looking to improve a robot's understanding of the world.
If you're successful, what difference will it make?
The ability for humans to explain the world around them to robots is huge: robots are fantastic at intensely complicated and quick computation, but poor on developing deep understanding. Humans are the opposite.
What are the risks and the payoffs?
We might be wasting our time/AI is dangerous/We might miss a better technique.
Payoffs are great -
How much will it cost?
For a system that already has humans to interact with, nothing.
How long will it take?
Depends on how complicated and exact it needs to be. We can propose models, or we can learn models from human experience.
What are the midterm and final "exams" to check for success?
How well does a human-robot team work with this form of communication?
Outline
- Introduction
- Related Work
- Generating SoftMax Models
- From Weights
- From Class Boundaries
- From Polytope Templates
- In n-dimensions
- Multimodal SoftMax
- Learning Spatial SoftMax Models
- From Collected Data
- With Prior Boundaries
- Using Symmetry to Minimize Data Collection
- Simulations
- Conclusion
Abstract
- Key words: Human-robot dialog, spatial language modeling and interpretation, Spatial-semantic mapping, Semantic perception
1. Introduction
Questions this section answers:
- What's the importance of a physical understanding of the world?
- How does it apply to a human-robot team?
- How can humans convey a physical understanding of the world to the robot?
- What types of physical understanding can we convey? How do we convey them?
- What domains does this apply to? When would it be used?
Questions this section answers:
- Who else has worked on communicating spatial understanding to a robot?
- How does our research differ from theirs?
- What have we worked on in the past, and how are we improving it?
- Nisar's previous work
- Nick Roy's Group (Tellex, Walter)
- Jensfelt and Burgard
- Dieter Fox, Cynthia Matuszek
- Marjorie Skubic (this?)
3. Generating SoftMax Models
Questions this section answers:
- What is a SoftMax Model and why does it best represent a spatial understanding?
- What different elements compose a SoftMax model, and what do they mean?
- What different ways can we construct a SoftMax model easily?
3.1. From Weights
Questions this section answers:
- What are the core insights we can get from SoftMax models?
- How is the slope of each class's probability derived and manipulated?
- How do manipulate (i.e. rotate/translate) the model?
3.2. From Class Boundaries
Questions this section answers:
- How do we compute SoftMax weights from class boundaries?
- When *can't* we use class boundaries?
3.3. From Polytope Templates
Questions this section answers:
- How do we derive normals from Polytope templates?
3.4. In n-dimensions
Questions this section answers:
- What do 1D examples looks like, and what purposes do they serve?
- Can we combine multiple states (i.e. position and velocty) into one SoftMax model?
3.5. Multimodal Softmax
Questions this section answers:
- Can we compose 'superclasses' from multiple SoftMax classes?
- Can we merge multiple state types (i.e. position and velocity) into a superclass?
- Can we fix the normal boundary problem using our classes as superclasses?
4. Learning SoftMax Models
Questions this section answers:
- Instead of specifying the SoftMax Models, can we learn them from user data?
- What use is collecting data from individuals? How different are they? Are humans good estimators?
- What language do humans use to express spatial representations? How many prepositions exist? Does their usage vary?
- Can we use negative information well?
4.1. From Collected Data
Questions this section answers:
- How do we learn distributions from experimental human data?
4.2. With Prior Boundaries
Questions this section answers:
- Given a gounding polytope, can we simply learn the gradients of the SoftMax classes?
- Can we represent the polytope boundaries probabilistically?
4.3. Using Symmetry to Minimize Data Collection
Questions this section answers:
- Can we minimize the amount of data needed to be collected by exploiting symmetry?
- Which prepositions or prepositional phrases imply symmetry?
5. Simulations
Questions this section answers:
- Is this SoftMax decomposition effective?
- Can it run in real-time?
- How did the robot perform on its own?
- How did the human-robot team perform when using positional statements? Velocity statements? Both?
- How useful was negative information?
6. Conclusion
Questions this section answers:
- What model did we introduce?
- What validation can we provide?
- What's next?