Publications

[Google Scholar] [DBLP]

Preprints

  • Beyond Variance Reduction: Understanding the True Impact of Baselines on Policy Optimization.
    Wesley Chung*, Valentin Thomas*, Marlos C. Machado, Nicolas Le Roux.
    CoRR abs/2008.13773; 2020. [PDF]

Journal Articles

  • Autonomous Navigation of Stratospheric Balloons using Reinforcement Learning.
    [Alph. order] Marc G. Bellemare, Salvatore Candido, Pablo S. Castro, Jun Gong, Marlos C. Machado, Subhodeep Moitra, Sameera Ponda, and Ziyu Wang.
    Nature; 588:77–82. 2020.
  • Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents.
    Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew Hausknecht, Michael Bowling.
    Journal of Artificial Intelligence Research (JAIR); 61: 523-562. 2018. [PDF]
  • True Online Temporal-Difference Learning.
    Harm van Seijen, A. Rupam Mahmood, Patrick M. Pilarski, Marlos C. Machado, Richard S. Sutton.
    Journal of Machine Learning Research (JMLR), 17(145): 1-40. 2016. [PDF]
  • RTSMate: Towards and Advice System for RTS Games.
    Renato L. de Freitas Cunha, Marlos C. Machado, Luiz Chaimowicz.
    Computers in Entertainment (CiE), 11(4): 1. 2014. [PDF]

Conference Papers

  • An Operator View of Policy Gradient Methods.
    Dibya Ghosh, Marlos C. Machado, Nicolas Le Roux.
    In Neural Information Processing Systems (NeurIPS); 2020. [PDF]
  • Count-Based Exploration with the Successor Representation.
    Marlos C. Machado, Marc G. Bellemare, Michael Bowling.
    In AAAI Conference on Artificial Intelligence (AAAI); 2020. [PDF] [Code]
  • Exploration in Reinforcement Learning with Deep Covering Options.
    Yuu Jinnai, Jee W. Park, Marlos C. Machado, George Konidaris.
    In International Conference on Learning Representations (ICLR); 2020. [PDF]
  • On Bonus Based Exploration Methods In The Arcade Learning Environment.
    Adrien A. Taiga, William Fedus, Marlos C. Machado, Aaron Courville, Marc G. Bellemare.
    In International Conference on Learning Representations (ICLR); 2020. [PDF]
  • Eigenoption Discovery through the Deep Successor Representation.
    Marlos C. Machado, Clemens Rosenbaum, Xiaoxiao Guo, Miao Liu, Gerald Tesauro, Murray Campbell.
    In International Conference on Learning Representations (ICLR); 2018. [PDF]
  • Accelerating Learning in Constructive Predictive Frameworks with the Successor Representation.
    Craig Sherstan, Marlos C. Machado, Patrick M. Pilarski.
    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); 2018. [PDF]
  • A Laplacian Framework for Option Discovery in Reinforcement Learning.
    Marlos C. Machado, Marc G. Bellemare, Michael Bowling.
    In International Conference on Machine Learning (ICML); 2017. [PDF] [Video] [Talk (15 min)] [Code]
  • State of the Art Control of Atari Games Using Shallow Reinforcement Learning.
    Yitao Liang, Marlos C. Machado, Erik Talvitie, Michael H. Bowling.
    In International Conference on Autonomous Agents and Multiagent Systems (AAMAS); 2016. [PDF] [Blog post] [Code]
  • Introspective Agents: Confidence Measures for General Value Functions.
    Craig Sherstan, Adam White, Marlos C. Machado, Patrick M. Pilarski.
    In Conference on Artificial General Intelligence (AGI); 2016. [PDF]
  • A Binary Classification Approach for Automatic Preference Modeling of Virtual Agents in Civilization IV.
    Marlos C. Machado, Gisele L. Pappa, Luiz Chaimowicz.
    In International Conference on Computational Intelligence and Games (CIG); 2012. [PDF]
  • Characterizing and Modeling Agents in Digital Games.
    Marlos C. Machado, Gisele L. Pappa, Luiz Chaimowicz.
    In Brazilian Symposium on Computer Games and Digital Entertainment (SBGames); 2012. [PDF]
  • Player Modeling: Towards a Common Taxonomy.
    Marlos C. Machado, Eduardo P. C. Fantini, Luiz Chaimowicz.
    In International Conference on Computer Games (CGames); 2011. [PDF]
  • Agents Behavior and Preferences Characterization in Civilization IV.
    Marlos C. Machado, Bruno S. L. Rocha, Luiz Chaimowicz.
    In Brazilian Symposium on Computer Games and Digital Entertainment (SBGames); 2011. [PDF]
  • Combining Metaheuristics and CSP Algorithms to Solve Sudoku.
    Marlos C. Machado, Luiz Chaimowicz.
    In Brazilian Symposium on Computer Games and Digital Entertainment (SBGames); 2011. [PDF]

Magazine Articles, Extended Abstracts, Workshops, and Others

  • Benchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment.
    Adrien A. Taiga, William Fedus, Marlos C. Machado, Aaron Courville, Marc G. Bellemare.
    ICML Workshop on Exploration in Reinforcement Learning; 2019. [Best paper award] [PDF]
    Longer version was published at ICLR’20.
  • Count-Based Exploration with the Successor Representation.
    Marlos C. Machado, Marc G. Bellemare, Michael Bowling.
    ICML Workshop on Exploration in Reinforcement Learning; 2018. [Best paper award] [PDF]
    Also presented at the Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM); 2019.
    Longer version was published at AAAI’20.
  • Generalization and Regularization in DQN.
    Jesse Farebrother, Marlos C. Machado, Michael Bowling.
    NeurIPS Deep Reinforcement Learning Workshop; 2018. [PDF]
    Also presented at the Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM); 2019.
  • Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents (Extended Abstract).
    Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew Hausknecht, Michael Bowling.
    In International Joint Conference on Artificial Intelligence (IJCAI); 2018. [Invited paper] [PDF]
  • The Eigenoption-Critic Framework.
    Miao Liu, Marlos C. Machado, Gerald Tesauro, Murray Campbell.
    NIPS Hierarchical RL Workshop; 2017. [PDF]
  • Learning Purposeful Behaviour in the Absence of Rewards.
    Marlos C. Machado, Michael Bowling.
    ICML Workshop on Abstraction in Reinforcement Learning; 2016. [PDF]
  • Domain-Independent Optimistic Initialization for Reinforcement Learning.
    Marlos C. Machado, Sriram Srinivasan, Michael Bowling.
    AAAI Workshop on Learning for General Competency in Video Games; 2015. [PDF]
  • Reports from the 2015 AAAI Workshop Program.
    Stefano V. Albrecht, J. Christopher L., David L. Buckeridge, Adi Botea, Cornelia Caragea, Chi-Hung Chi, Theodoros Damoulas, Bistra N. Dilkina, Eric Eaton, Pooyan Fazli, Sam Ganzfried, Marius Thomas Lindauer, Marlos C. Machado, Yuri Malitsky, Gary Marcus, Sebastiaan Meijer, Francesca Rossi, Arash Shaban-Nejad, Sylvie Thiébaux, Manuela M. Veloso, Toby Walsh, Can Wang, Jie Zhang, Yu Zheng.
    AI Magazine; 36(2): 90-101, 2015. [PDF]

Theses

  • Efficient Exploration in Reinforcement Learning through Time-Based Representations
    Marlos C. Machado.
    Ph.D. Thesis, Computing Science Department, University of Alberta, Edmonton, AB, Canada; 2019. [PDF]
  • A Methodology for Player Modeling based on Machine Learning
    Marlos C. Machado.
    M. Sc. Thesis, Computer Science Department, Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brazil.; 2013. [PDF]
Marlos C. Machado