Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Interesting - I've heard this anecdotally. Curious if you know of any resources that look at this in more detail?


I haven't seen any papers doing a proper analysis on the topic, just mostly saying this from firsthand experience testing a handful of them and comparing to the model they were based on given same prompt and sampler. It's usually not even close and you can immediately tell that it's notably dumber. Iirc in one case one even forgot how to do basic arithmetic while the original model aced it. Not entirely unexpected results from sticking a digital ice pick into the weights.

Afaik there are only three major sources of quality unaligned model versions, which are Nous's Hermes models, Hartford's Dolphins and Drummer's Tigers. All of them regular fine tunes that are mostly the same or just ever so slightly lower in performance as the original.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: