We use the concept of 'topics'. A topic encodes: 'The part of the index a query cares about'.

For example, given a query like "fetch user where id = 1", there could be a topic like "users:id:1" [1]

When a WAL record comes in, we find queries by their topics, and invalidate them. This triggers a refresh.

This is inspired by Figma's LiveGraph [2], which in turn is inspired by Asana's Luna [3]. The essays cover the idea in more detail, but you can also start diving into the code, at invalidator.clj [4]

[1] This is an example. Inside Instant, we encode them as a 'datalog' pattern. Something like `[:ea "1" :users/id]` [2] https://blog.asana.com/2020/09/worldstore-distributed-cachin... [3] https://www.figma.com/blog/livegraph-real-time-data-fetching... [4] https://github.com/instantdb/instant/blob/main/server/src/in...

I worked on LiveGraph for a long time at Figma. We went through our own evolution:

1. first we would use the WAL records to invalidate queries that could be affected (with optimizations for fast matching) and requery the data

2. then we used the info from the WAL record to update the query in-memory without asking the DB for the new result, it worked for majority of the queries that can be reliably modeled outside of the DB

3. I believe after I left the team reverted to the re-query approach, as managing a system that replicates the DB behavior was not something they were excited to maintain, and as the DB layer got scaled out, extra DB queries were less of a downside

LiveGraph was an inspiration for us Slava, made me smile to see your comment. 2. is _really_ interesting.

I'll reach out to you on twitter; would love to learn more about your experience