A team at Uber's artificial intelligence lab has developed an algorithm that achieved high scores in the classic Atari games "Pitfall!" and "Montezuma's Revenge," MIT Technology Review reports. The research, detailed in an upcoming paper and a Uber blog post, addresses a long-standing challenge in AI: games where rewards are not immediately apparent.
Unlike games such as "Super Mario Bros.," where collecting coins or defeating enemies provides instant feedback, "Pitfall!" and "Montezuma's Revenge" require players to execute long sequences of actions before any score increase. This structure has stymied reinforcement learning algorithms, which learn by seeking programmed rewards. Even algorithms designed to explore randomly struggled to identify the steps needed to progress.
Uber's algorithm incorporates a memory mechanism that allows it to learn from past attempts, improving its ability to navigate these complex environments. However, it still required human supervisors to suggest promising areas or strategies, according to the blog post. With this hybrid approach, the algorithm ultimately surpassed human-level scores.
The findings suggest that for certain complex problems, human-AI collaboration may be more effective than fully autonomous systems. As the research indicates, high scores in challenging games—and potentially other difficult tasks—may be achieved through such partnerships rather than by machines alone.
Editorial Notes