I'm facing the problem you describe daily. It's especially bad because it's very difficult for me to predict if the set of filters will reduce the dataset by ~1% (in which case following the original vector index is fine) or by 99.99% (in which case you just want to brute force the remaining vectors).
Tried a million different things, but haven't heard of Turbopuffer yet. Any references on how they perform with such additional filters?
Lucene and ES implement a shortcut for filters that are restrictive enough. Since it's already optimized for figuring out if something falls into your filter set, you first determine the size of that. You traverse the HNSW normally, then if you have traversed more nodes than your filter set's cardinality, you just switch to brute forcing your filter set distance comparisons. So worst case scenario is you do 2x your filter set size vector distance operations. Quite neat.
Oh that's nice! Any references on this shortcut? How do you activate that behavior? I was playing around with ES, but the only suggestion I found was to use `count` on filters before deciding (manually) which path to take.
I have a book which lists companies which sent out mail order catalogs back in the 70s --- I'm pretty sure it was listed in a Whole Earth Catalog (might even have been published by them).
> The first thing you notice when you land in Greenland is there are no trees or grass. There is snow and then there is exposed rock.
This is only true of the area around the airport. Even his pictures further into the article show how misleading this description is. I was actually very surprised how little snow/ice there was. Now when I think of Greenland, I think of something similar to [1].
Of course, in the winter, it's a completely different story (I was there in July). But he was there during the warm period as well (as is obvious from his photos).
> The city itself sits in a landscape so dramatically inhospitable it makes the surface of Mars look cozy.
If you look at a map, you will notice that Nuuk is at the same latitude as Reykjavik. There's a common meme about Iceland being green and Greenland being icy, and that's definitely true if you compare inland or northern Greenland with Iceland during summer (during winter, both are icy and dark), but hiking around Nuuk is a very "green" experience. Yes, there's a ton of mosquitoes, but nature itself is very inviting. I did not get any of the "inhospitable" vibes he mentions.
> But again even riding the bus around it is impossible to escape the feeling that this is a fundamentally hostile to human life place. The sun is bright and during the summer its pretty hot, with my skin feeling like it was starting the burn pretty much the second it was exposed to the light. It's hard to even dress for, with layers of sunscreen, bug spray and then something warm on top if you suddenly got cold.
This whole section is just overblown BS.
All in all, I enjoyed it a lot. Compared to Iceland, it's definitely a lot less "user friendly" and you need to prepare better, but I have never been to a place that is less affected by humans, and in our age, that is something worth experiencing.
>> The first thing you notice when you land in Greenland is there are no trees or grass. There is snow and then there is exposed rock.
> This is only true of the area around the airport. Even his pictures further into the article show how misleading this description is.
At least as far as trees go, Greenland is reasonably famously lacking in trees (if you are the kind of person who cares about such things). All chopped down by the Vikings and only now are a few sections of forest being regrown. Iceland is basically the same.
I tried explaining in the other comment here. In summary, it's beautiful, raw nature, that's different from anything else I've seen. It takes some preparation, but I haven't experience any of these "sun will kill you and ice will kill you too" vibes that the article suggests. It felt like one of the last places on Earth not affected by humans (at least directly through resource exploitation, as it definitely is affected by the warming - but that's another discussion).
Exactly. I tend to like Hotz, but by his description, every developer is also "a compiler", so it's a useless argument.
My life quality (as a startup cofounder wearing many different hats across the whole stack) would drop significantly if Cursor-like tools [1] were taken away from me, because it takes me a lot of mental effort to push myself to do the boring task, which leads to procrastination, which leads to delays, which leads to frustration. Being able to offload such tasks to AI is incredibly valuable, and since I've been in this space from "day 1", I think I have a very good grasp on what type of task I can trust it to do correctly. Here are some examples:
- Add logging throughout some code
- Turn a set of function calls that have gotten too deep into a nice class with clean interfaces
- Build a Streamlit dashboard that shows some basic stats from some table in the database
- Rewrite this LLM prompt to fix any typos and inconsistencies - yeah, "compiling" English instructions into English code also works great!
- Write all the "create index" lines for this SQL table, so that <insert a bunch of search usecases> perform well.
[1] I'm actually currently back to Copilot Chat, but it doesn't really matter that much.
That's one of the thing that I wouldn't delegate to LLM. Logging is like a report of things that happens. And just like a report, I need relevant information and the most useful information.
...
A lot of these use cases actually describes the what. But the most important questions is always the why. Why is it important to you? Or to the user? That's when things have a purpose and not be just toys.
Code with logging is "self reporting". Adding logging statements is not reporting itself. Adding `logger.error(f"{job} failed")` is not reporting itself, and LLMs are perfectly capable of adding such statements in applicable places.
As to why, it's because I'm building an app with a growing userbase and need to accommodate to their requirements and build new features to stay ahead of the competition. Why you decided I'm describing a toy project is beyond me.
As someone else said: Launch is only the first step. That's when practicality start to matter.
The reason senior engineers are being paid that well is not because they need to type a lot of code to get new features in. It's because they need to figure how to have less code while having more features.
In the context compression approach, why aren't the agents labeled as subagents instead? The compressed context is basically a "subtask".
This is my main issue with all these agentic frameworks - they always conviniently forget that there is nothing "individual" about the thing they label "an agent" and draw a box around.
Such "on demand" agents, spawned directly from previos LLM output, are never in any way substantially different from dynamic context compression/filtering.
I think the only sensible framework is to think in terms of tools, with clear interfaces, and a single "agent" (single linear interaction chain) using those tools towards a goal. Such tools could be LLM-based or not. Forcing a distinction between a "function tool" and an "agent that does somethng" doesn't make sense.
In my experience it does not work better. There are two context related values to subagent tool calls: (1) the subagent trials and deliberations don’t poison the callers context [this is a win here]; and (2) the called agent isn’t unduly influenced by the caller’s context. [problem!]
The latter is really helpful for getting a coding assistant to settle on a high quality solution. You want critic subagents to give fresh and unbiased feedback, and not be influenced by arbitrary decisions made so far. This is a good thing, but inheriting context destroys it.
Not OP, but I have long thought of this type of approach (underlying "hard coded" object tracking + fuzzy AI rendering) to be the next step, so I'll respond.
The problem with using equations is that they seem to have plateaued. Hardware requirements for games today keep growing, and yet every character still has that awful "plastic skin", among all the other issues, and for a lot of people (me included) this creates heavy uncanny-valley effects that makes modern games unplayable.
On the other hand, images created by image models today look fully realistic. If we assume (and I fully agree that this is a strong and optimistic assumption) that it will soon be possible to run such models in real time, and that techniques for object permanence will improve (as they keep improving at an incredible phase right now), then this might finally bring us to the next level of realism.
Even if realism is not what you're aiming for, I think it's easy to imagine how this might change the game.
You're comparing apples to oranges, holding up today's practical real-time rendering techniques against a hypothetical future neural method that runs many orders of magnitude faster than anything available today, and solves the issues of temporal stability, directability and overall robustness. If we grant "equation based" methods the same liberty then we should be looking ahead to real-time pathtracing research, which is much closer to anything practical than these pure ML experiments.
That's not to say ML doesn't have a place in the pipeline - pathtracers can pair very well with ML-driven heuristics for things like denoising, but in that case the underlying signal is still grounded in physics and the ML part is just papering over the gaps.
The question was "why does it feel more real", and I answered that - because the best AI generated images today feel more real than the best 3D renders, even when they take all the compute in the world to finish. So I can imagine that trend going forward into real-time rendering as well.
I did not claim that AI-based rendering will overcome traditional methods, and have even explicitly said that this is a heavy assumption, but explained why I see it as exciting.
I think we'll have to agree to disagree about well done 3D renders not feeling real. Movie studios still regularly underplay how much CGI they use for marketing purposes, and get away with it, because the CGI looks so utterly real that nobody even notices it until much later when the VFX vendors are allowed to give a peek behind the curtain.
e.g. Top Gun Mavericks much lauded "practical" jet shots, which were filmed on real planes, but then the pilots were isolated and composited into 100% CGI planes with the backdrops also being CGI in many cases, and huge swathes of viewers and press bought the marketing line that what they saw was all practical.
I find it odd that you're that bothered by uncanny valley effects from game rendering but apparently not by the same in image model outputs. They get little things wrong all the time and it puts me off the image almost instantly.
>Hardware requirements for games today keep growing, and yet every character still has that awful "plastic skin", among all the other issues
That's because the number of pixels to render onto keep growing. Instead of focusing on physically based animations and reactions, we chose to leap from 480p to 720p overnight, and then to 1080p in a few more years. Now we quadrupled that and want things with more fidelity with 4x the resolution of last generation.
> images created by image models today look fully realistic.
Because they aren't made in real time (I'll give the BOTD for now and say theya are "fully realistic". Even this sample here claims 100ms. Rendering at 6-7 seconds per frame isn't going to work for any consumer product at any point in gaming history.
>Even if realism is not what you're aiming for, I think it's easy to imagine how this might change the game.
not in real time rendering. I am interested to see if this can help with offline stuff, but we're already so strapped for performance withoout needing to query an "oracle" between frames.
as of now, I'm not even that convinced by the Nvidia 5X series of frame interpolation (which would only be doable by hardware manufactureres).
Note that (in the first test, the only one where output length is reported), Gemini Pro returned more than 3x the amount of text, at less than 2x the amount of time. From my experience with Gemini, that time was probably mainly spent on thinking, length of which is not reported here. So looking at pure TPS of output, Gemini is faster, but without clear info on the thinking time/length, it's impossible to judge.
Anyone who has recently worked on embedding model finetuning, any useful tools you'd recommend (both for dataset curation and actual finetuning)? Any models you'd recommend as especially good for finetuning?
I'm interested in both full model finetunes, and downstream matrix optimization as done in [1].
Tried a million different things, but haven't heard of Turbopuffer yet. Any references on how they perform with such additional filters?