Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

AFAIK (not much) it definitely helps to train on longer sequences even with rope/yarn and is needed if you care about long context performance (and not just the long context capability).


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: