Methodology
The next chapter to tackle is the methodology. To be completely transparent, as of writing this I already wrote the whole codebase and started running some of the experiments. I thought this way I would have an easier time writing this chapter. As before I wanted to share the structure and rough first draft.
Background
Benchmark Design
- In this subsection I should first callback to the main problem (attribution of change-type to forgetting) and then list the overall design principles as well as the whole pipeline / architecture of the benchmark runs.
- CRL baselines and the common CLMethod interface (reference 2.3.2 for the more detailed mechanisms and NOT re-explain them)
- Reference back to JAXtari framework and modification system (transition to "Task Construction").
Task Construction
- Mod taxonomy -> visual, dynamic and reward mapped onto O, P and R
- Observatoin modalities as a design dimension (mention RQs) -> pixel (RGB) vs OC)
- Sequence Families (vis4, dyn4, rew4 -> five tasks each, base + 4 mods) and the magnitude mods for RQ3 + the task ordering (select random order once, keep fixed)
- Game selection based on two criteria: speed and similarity to ALE
- Design Invariants and Constraints
- Representative example of mods as well as visuals of the mods
Evaluation and Metrics
- Continual train/eval protocol -> the loop, and details ...
- Why PPO?
- Loop: Init Agent → train on base task → savecheckpoint → keeps on training on modded task variants (4 per sequence) → after each train: eval phase (and during training on mods the agents still gets evaluated every X steps on the base env)
- Metrics: Forgetting / Retention, per-task recency weighted forgetting, clamping to [0,1] etc.
Get in Touch
If you have any feedback you want to share with me feel free to reach out at mail@sebastianwette.de. I would be more than happy to chat about it.