After my time at Brokk I picked up this idea and developed Casefile. To avoid loading the entire previous session context, the agent creates a task log. So a follow-up task only needs to load the necessary task logs + git state as the base to implement its task. Each log is linked to its commit, so git blame on a line leads back to the decision behind it. Another advantage of this approach is that you can retrieve the intent why a code change was made and which alternative solutions were skipped. You can see the workflow in the open at https://github.com/native-federation/devtools.
Last month I extended the system to store the plan and task logs in a non-public Casefile repository because my enterprise customers don't allow work artifacts like these in their repositories.
Give it a shot if you think it's useful. I would like to hear your feedback!