The subtle difference is between American parking minimums imposed on property owners - “you must reserve space on your private property for this many cars whether you own them or not” vs Japanese parking requirements imposed on car owners - “you must reserve space on some private property for your car if you want to own it”
Considerably more in many cases. The point of floating point is to have as many distinct values in the range 2-4 as are in the range 1-2 as are between 1/2 and 1, 1/4 and 1/2, 1/8 and 1/4, etc. the smallest representable difference between consecutive floating point numbers down around the size of 1/64 is on the order of epsilon/64
Multiplying epsilon by the largest number you are dealing with is a strategy that makes using epsilons at least somewhat logical.
I’m not sure ‘patched’ is the right word here. Are you suggesting they edited the LLM weights to fix cabbage transportation and car wash question answering?
Absolutely not my area of expertise but giving it a few examples of what should be the expected answer in a fine-tuning step seems like a reasonable thing and I would expect it would "fix" it as in less likely to fall into the trap.
At the same time, I wouldn't be surprised if some of these would be "patched" via simply prompt rewrite, e.g. for the strawberry one they might just recognize the question and add some clarifying sentence to your prompt (or the system prompt) before letting it go to the inference step?
But I'm just thinking out loud, don't take it too seriously.
https://docs.unity3d.com/560/Documentation/Manual/SL-Shader....
reply