Oh yeah, this is huge! I instruct agents to do a few things in this vein that are big improvements:
1. Have agents emit chatter in a structured format. Have them emit hypotheses, evidence, counterfactuals, invariants, etc. The fully natural language agent chatter is shit for observability, and if you have structured agent output you can actually run script hooks that are very powerful in response to agent input/output.
2. Have agents summarize key evidence from toolcalls, then just drop the tool call output from context (you can give them a tool to retrieve the old value without recomputation, cache tool output in redis and give them a key to retrieve it later if needed). Tool calls dominate context expansion bloat, and once you've extracted evidence the original tool output is very low value.
1. Have agents emit chatter in a structured format. Have them emit hypotheses, evidence, counterfactuals, invariants, etc. The fully natural language agent chatter is shit for observability, and if you have structured agent output you can actually run script hooks that are very powerful in response to agent input/output.
2. Have agents summarize key evidence from toolcalls, then just drop the tool call output from context (you can give them a tool to retrieve the old value without recomputation, cache tool output in redis and give them a key to retrieve it later if needed). Tool calls dominate context expansion bloat, and once you've extracted evidence the original tool output is very low value.