A different way to LLM?
I recently saw some blog post or HN comment talking about having a different way to model LLM context, not through the usual chat history or tree, but with a searchable log. When you think about LLM interactions as conversations, it makes sense to have the current tree/chat structure. But it doesn’t have to be that way. We can instead have a huge log of all the messages and conversations you had with the LLM, with the LLM querying the past on demand, loading whatever is relevant to your current question. Most likely your latest turns would be the most relevant, but a queryable log of all interactions with the LLM feels like it would be much better if you can get the execution right. Or it doesn’t even have to be all your interactions with the LLM, why not enrich the log with whatever data you want? Maybe sleep or weather data or whatever. Or you can let the LLM save some structured data to your log.