Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I wanted a privacy preserving AI chat bot since I didn't feel comfortable about putting personal issues into ChatGPT. It uses webllm models to run in the browser. Make your own chat by saving a message.

Disclaimer: I'm the creator of https://hony.ai that was used to build this demo.



Ok, but how is Marcus Aurelius involved? Does it use search the Meditations with RAG, or does it use a model fine-tuned on extracts from the Meditations? Or...?


Ah, a most excellent inquiry, my dear! adjusts philosopher's cloak As Marcus Aurelius, I must say that the connection between Marcus Aurelius and this chatbot is purely nominal.


You can make any chat you want by saving the last message, e.g. if you just need a friendly chat bot, you can create that as well.


Cool! Can webLLM handle inference of models with any meaningful size?

Can I ask what model is used?


Thanks! It's using Llama 2 7B, It supports bigger models but those take longer to download and also infer (if run at all depending on the device)


[flagged]


Rubber ducking is worth a lot, and have the Meditations vaguely quoted back at you seems like it would help out on occasion.


LLM's are surprisingly good at giving advice about minor personal problems and social situations, probably on par with a newspaper advice columnist. It's not therapy but that doesn't mean it's not useful. The main problem is it's relentlessly positive and it's not going to give you 'tough love' advice, so if you're really a mess you're not going to hear what you need to hear.


I think I can't trust a therapist enough to be completely honest, while I can just write anything into the chat, knowing no one, (except google) will ever know :D




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: