ACCEL: Evolving Curricula with Regret-Based Environment Design (Paper Review)
#ai #accel #evolution
Automatic curriculum generation is one of the most promising avenues for Reinforcement Learning today. Multiple approaches have been proposed, each with their own set of advantages and drawbacks. This paper presents ACCEL, which takes the next step into the direction of constructing curricula for multi-capable agents. ACCEL combines the adversarial adaptiveness of regret-based sampling methods with the capabilities of level-editing, usually found in Evolutionary Methods.
OUTLINE:
0:00 - Intro & Demonstration
3:50 - Paper overview
5:20 - The ACCEL algorithm
15:25 - Looking at the pseudocode
23:10 - Approximating regret
33:45 - Experimental results
40:00 - Discussion & Comments
Website: https://accelagent.github.io
Paper: https://arxiv.org/abs/2203.01302
Abstract:
It remains a significant challenge to train generally capable agents with reinforcement learning (RL). A promising avenue for improving the robustness of RL agents is through the use of curricula. One such class of methods frames environment design as a game between a student and a teacher, using regret-based objectives to produce environment instantiations (or levels) at the frontier of the student agent's capabilities. These methods benefit from their generality, with theoretical guarantees at equilibrium, yet they often struggle to find effective levels in challenging design spaces. By contrast, evolutionary approaches seek to incrementally alter environment complexity, resulting in potentially open-ended learning, but often rely on domain-specific heuristics and vast amounts of computational resources. In this paper we propose to harness the power of evolution in a principled, regret-based curriculum. Our approach, which we call Adversarially Compounding Complexity by Editing Levels (ACCEL), seeks to constantly produce levels at the frontier of an agent's capabilities, resulting in curricula that start simple but become increasingly complex. ACCEL maintains the theoretical benefits of prior regret-based methods, while providing significant empirical gains in a diverse set of environments. An interactive version of the paper is available at this http URL.
Authors: Jack Parker-Holder, Minqi Jiang, Michael Dennis, Mikayel Samvelyan, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yann...
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannick...
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
-
1:01:20
In The Litter Box w/ Jewels & Catturd
19 hours agoSCHIFFSHOW | In the Litter Box w/ Jewels & Catturd - Ep. 591 - 6/20/2024
29.8K28 -
20:44
Breaking Points
8 hours agoCNN Blames 'FAKE VIDEOS' For Biden Age Concern
21.7K20 -
2:24:05
Tucker Carlson
4 hours agoNeil Oliver: How Banks Took Over Empires, and the Truth About WWII, Brexit, & COVID
92.9K176 -
48:59
The Casey Adams Show
3 days agoDavid Sacks & Evan Owen - Glue: The AI Solution to Workplace Communication | The Casey Adams Show
36.7K6 -
1:00:51
LFA TV
10 hours agoMILES TRIAL DAY 20 | 6.20.24 3PM EST
17.2K2 -
1:53:45
Barstool Yak
4 hours agoThe Yak with Big Cat & Co. Presented by Rhoback | The Yak 6-20-24
20.6K55 -
49:29
Ben Shapiro
5 hours agoEp. 1988 - Bring BACK The Ten Commandments in Schools
43.7K83 -
1:01:13
Russell Brand
8 hours agoEXPOSED: Pentagon Ran Anti-Vax Campaign In Asia While CENSORING YOU! You Won’t F*cking Believe This!
134K190 -
1:57:37
The Charlie Kirk Show
5 hours agoWhat Polls To Trust? + False Flags? | Boyle, Beattie | 6.20.24
78.9K53 -
1:47:56
TheAlecLaceShow
6 hours agoGuests: Charlie Kirk | Valentina Gomez | Ten Commandments in the Classroom | The Alec Lace Show
36K10