 |
Thomas Walsh
thomasjwalsh [ at ] gmail {dot} com |
Research and Current Position
I am currently a Senior Research Scientist at Sony AI, exploring new applications of cutting-edge AI and reinforcement learning techniques. Previously, I worked in several academic and industry jobs investigating AI, data science, and sequential decision making techniques including leading a team of researchers at Kronos Incorporated (now UKG), and holding research positions at MIT, The University of Kansas, The University of Arizona, and Rutgers University (as one of Michael Littman's students), and my undergrad research at UMBC. You can find publications from my research below or for a quick tour here is a copy of my CV .
Publications
Journal Articles
Varun Kompella, Thomas J. Walsh, Samuel Barrett, Peter R. Wurman, Peter Stone Event Tables for Efficient Experience Replay Transactions on Machine Learning Research (TMLR), 2023.
Peter R. Wurman, Samuel Barrett, Kenta Kawamoto, James MacGlashan, Kaushik Subramanian, Thomas J. Walsh, Roberto Capobianco, Alisa Devlic, Franziska Eckert, Florian Fuchs, Leilani Gilpin, Piyush Khandelwal, Varun Kompella, HaoChih Lin, Patrick MacAlpine, Declan Oller, Takuma Seno, Craig Sherstan, Michael D. Thomure, Houmehr Aghabozorgi, Leon Barrett, Rory Douglas, Dion Whitehead, Peter Dürr, Peter Stone, Michael Spranger, Hiroaki Kitano Outracing champion Gran Turismo drivers with deep reinforcement learning Nature 602 223-228, 2022.
Robert C. Grande, Thomas J. Walsh, Girish Chowdhary, Sarah Ferguson, Jonathan P. How. Online Regression for Data with Changepoints using Gaussian Processes and Reusable Models IEEE Transactions on Neural Networks and Learning Systems, 2016.
- Note: Lemma 1 in the GP paper linked above makes the unstated assumption that a GP is an unbiased estimator of the mean of the target function. In cases where a bias is introduced, an additional bias error term would be added to the result.
Mark Cutler, Thomas J. Walsh, Jonathan P. How. Real-World Reinforcement Learning via Multi-Fidelity Simulators. IEEE Transactions on Robotics (TRO), 2015.
Bernard Michini, Thomas J. Walsh, Ali-akbar Agha-mohammadi, and Jonathan P. How.Bayesian Nonparametric Reward Learning from Demonstration IEEE Transactions on Robotics (TRO), 2015.
Girish Chowdhary, Miao Liu, Robert C. Grande, Thomas J. Walsh, Jonathan P. How and Lawerence Cairn. "Off-Policy Reinforcement Learning with Gaussian Processes". Accepted in Acta Automatica Sinica, 2014.
Alborz Geramifard, Thomas J. Walsh, Stefanie Tellex, Girish Chowdhary, Nicholas Roy, and Jonathan P. How. A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning Foundations and Trends in Machine Learning (FTML), 2013.
Thomas J. Walsh, Michael L. Littman, and Alexander Borgida Learning Web-Service Task Descriptions from Traces Web Intelligence and Agent Systems, Volume 10, Number 4, (pages 397-421), 2012.
Lihong Li, Michael L. Littman, Thomas J. Walsh, and Alexander L. Strehl Knows what it knows: A framework for self-aware learning. Machine Learning, Volume 82, Number 3, (pages 399-443), 2011.
Fusun Yaman, Thomas J. Walsh, Michael L. Littman, Marie desJardins Democratic Approximation of Lexicographic Preference Models Artificial Intelligence, Special Issue on Representing, Processing, and Learning Preferences, Volume 175 (pages 1290-1307), 2011.
Thomas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman Learning and Planning in Environments with Delayed Feedback In the Journal of Autonomous Agents and Multi-Agent Systems, Volume 18, Issue1, (pages 83-101), February, 2009.
Dennis D.Y. Kim, Thomas T.Y. Kim, Thomas Walsh, Yoshifumi Kobayashi, Tara C. Matise, Steven Buyske, and Abram Gabriel
Widespread RNA Editing of Embedded Alu Elements in the Human Transcriptome
Genome Res. 2004 14 (September): 1719-1725.
Conference Papers
Dustin Morrill, Thomas J. Walsh, Daniel Hernandez, Peter R. Wurman, Peter Stone. Composing Efficient, Robust Tests for Policy Selection In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI-23), 2023.
Robert C. Grande, Thomas J. Walsh, Jonathan P. How. Sample Efficient Reinforcement Learning with Gaussian Processes In Proceedings of the International Conference on Machine Learning (ICML-14), Beijing, China, 2014.
Appendix available here
- Note: Lemma 1 in the GP paper linked above makes the unstated assumption that a GP is an unbiased estimator of the mean of the target function. In cases where a bias is introduced, an additional bias error term would be added to the result.
Mark Cutler, Thomas J. Walsh, Jonathan P. How. Reinforcement Learning with Multi-Fidelity Simulators In Proceedings of the International Conference on Robotics and Automation (ICRA-14), Hong Kong, 2014.
Alborz Geramifard, Thomas J. Walsh, Nicholas Roy, and Jonathan P. How. Batch-iFDD for Representation Expansion in Large MDPs In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI-13), Bellevue, WA, 2013.
Thomas J. Walsh and Sergiu Goschin. Dynamic Teaching in Sequential Decision Making Environments In Proceedings of theConference on Uncertainty in Artificial Intelligence (UAI-12), Catalina, CA, 2012.
Appendix available here.
Thomas J. Walsh, Daniel Hewlett, and Clayton T. Morrison Blending Autonomous Exploration and Apprenticeship Learning In Proceedings of the Twenty-Fifth Annual Conference on Neural Information Processing Systems (NIPS-11), Granada, Spain, 2011.
Also appeared at the 2011 RSS Workshop on the State of Imitation Learning.
Derek T. Green, Thomas J. Walsh, Paul R. Cohen and Yu-Han Chang. Learning a Skill-Teaching Curriculum with Dynamic Bayes Nets In Proceedings of the Twenty-Third Conference on Innovative Applications of Artificial Intelligence (IAAI-11), San Francisco, CA, 2011.
Daniel Hewlett, Thomas J. Walsh, and Paul R. Cohen. Teaching and Executing Verb Phrases In Proceedings of the First Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-Epirob-11), Frankfurt, Germany, 2011.
Also appeared at the RSS Workshop on the State of Imitation learning and the AAAI Spring Symposium on Bridging the Gaps in Human-Agent Collaboration.
Raquel Torres Peralta, Tasneem Kaochar, Ian R. Fasel, Clayton T. Morrison, Thomas J. Walsh, Paul R. Cohen. Challenges to Decoding the Intention Behind Natural Instruction In Proceedings of the IEEE International Symposium on Robots and Human Interactive Communications (RO-MAN-2011), Atlanta, GA, 2011.
Derek T. Green, Thomas J. Walsh, Paul R. Cohen, Carole R. Beal and Yu-han Chang. "Gender Differences and the Value of Choice in Intelligent Tutoring Systems". In Proceedings of User Modeling, Adaptation and Personalization (UMAP-2011), Girona, Spain, 2011.
Tasneem Kaochar, Raquel Torres Peralta, Ian R. Fasel, Clayton T. Morrison, Thomas J. Walsh, Paul R. Cohen Towards Understanding How Humans Teach Robots In Proceedings of User Modeling, Adaptation and Personalization (UMAP-2011), Girona, Spain, 2011.
Versions also appeared at the AAAI Spring Symposium on Bridging the Gaps in Human-Agent Collaboration and the 2011 Workshop on Agents Learning Interactively from Human Teachers (ALIHT) at IJCAI-2011.
Thomas J. Walsh, Sergiu Goshin, and Michael L. Littman Integrating Sample-based Planning and Model-based Reinforcement Learning In Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10), Atlanta, GA, 2010.
Thomas J. Walsh, Kaushik Subramanian, Michael L. Littman, and Carlos Diuk Generalizing Apprenticeship Learning across Hypothesis Classes In Proceedings of the Twenty-Seventh International Conference on Machine Learning (ICML-10), Haifa, Israel, 2010.
Also appeared at the 2010 Workshop on Agents Learning Interactively from Human Teachers (ALIHT) at AAMAS-10.
Thomas J. Walsh, István Szita, Carlos Diuk, and Michael L. Littman Exploring Compact Reinforcement-Learning Representations
with Linear Regression In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI-09), Montreal, Quebec, 2009.
A Tech Report is available for this paper that corrects the bounds reported in the conference version (with full proofs).
Thomas J. Walsh and Michael L. Littman Efficient Learning of Action Schemas and Web-Service Descriptions In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI-08), Chicago, IL, 2008.
Expanded version available as a Technical Report
Lihong Li, Michael L. Littman, Thomas J. Walsh Knows What It Knows: A Framework for Self-Aware Learning In Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML-08), Helsinki, Finland, 2008.
Co-winner of the ICML 2008 Best Student Paper Award
Fusun Yaman, Thomas J. Walsh, Michael L. Littman, Marie desJardins Democratic Approximation of Lexicographic Preference Models In Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML-08), Helsinki, Finland, 2008.
Also appeared at the 4th Multidisciplinary Workshop on Advances in Preference Handling at AAAI-08.
Thommas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman Planning and Learning in Environments with Delayed Feedback In Proceedings of the 18th European Conference on Machine Learning (ECML-07),
Warsaw, Poland, 2007.
Lihong Li, Thomas J. Walsh, and Michael L. Littman Towards a Unified Theory of State Abstraction for MDPs Proceedings of the Ninth International Symposium on Artificial Intelligence and Mathematics (AIMA06), Ft. Lauderdale, FL, 2006.
Bethany R. Leffler, Michael L. Littman, Alexander L. Strehl, Thomas J. Walsh. Efficient Exploration With Latent Structure In Proceedings of Robotics: Science and Systems. Cambridge, Massachusetts, 2005.
Thesis
Thomas J. Walsh. Efficient Learning of Relational Models for Sequential Decision Making (also available without hyperlinks).
Other Publications (Workshops, Magazines, Special Publications)
Thomas J. Walsh, Javad Taheri, Jeremy B. Wright and Paul R. Cohen. Leadership Games and their Application in Super-Peer Networks AAAI Workshop on Applied Adversarial Reasoning and Risk Modeling, San Francisco, CA, 2011.
Thomas J. Walsh and Michael L. Littman Planning with Conceptual Models Mined from User Behavior In Proceedings of the AAAI-07 Workshop on Acquiring Planning Knowledge via Demonstration, Vancouver, BC, 2007.
Thomas J. Walsh, Lihong Li, and Michael L. Littman Transferring State Abstractions Between MDPs In Proceeding of the ICML-06 Workshop on Structural
Knowledge Transfer for Machine Learning, Pittsburgh, PA, 2006.
Alex Borgida, Thomas J. Walsh, and Haym Hirsh. Towards Measuring Similarity in Description Logics In Proceedings of the 2005 International Workshop on Description Logics (DL2005), Edinburgh, Scotland, 2005.
Thomas J. Walsh and D. Richard Kuhn. Challenges in Securing Voice over IP IEEE Security & Privacy. Vol 3(3) 2005 (May/June) : 44-49.
D. Richard Kuhn, Thomas J. Walsh, Steffen Fries.
Security Considerations for Voice Over IP Systems Special Publication from the National Institute of Standards and Technology, 2005 [final version!] (slash-dotted on 5/6/2005 here).
Here's a video of RL in action (implemented with the help of many others in the RL^3 Lab back in 2005) on a Sony Aibo trying to escape from a darkened room. It is pretty dated by today's standards, but exhibits some interesting core concepts. Video
A webpage with more info on the task is here: Explanation