I recall there was a website tracking the ideological bias of LLMs, but I can’t find it now. But it was showing where all the LLMs rank on a political graph with four quadrants. I think we need something like that, ranking these LLMs on aspects like censorship.

Example: https://www.technologyreview.com/2023/08/07/1077324/ai-langu...

But I think some other site was doing this ‘live’ and adding more models as they appeared.

Do you mean this? https://trackingai.org/

Yes, that is exactly what I was thinking of. Thank you.

It's likely RAG / augmented with web data. Would be interested if local execution returned the same results.

It is. You can see a little "G" icon indicating that it searched the web with Google.

Interesting, mine has a different icon and used Bing

It certainly doesn't, I'm running the 7B locally with ollama

It provided a lot more detail about the case, but does not have current information. It hallucinated the question about juror count, or maybe confused it with a different case seems more likely, one of the E Jean Carroll cases or the SDNY Trump Org financial fraud case?

You: how many jurists have been selected in the Trump trial in New York?

Meta AI: A full jury of 12 people has been selected for former President Donald Trump's trial in New York City, in addition to one alternate ¹. The selection process will continue in order to select five more alternates, though it is hoped that the selection process will be finished tomorrow ². Once all alternates have been selected, opening statements can begin ¹.

--- https://www.nbcnews.com/politics/donald-trump/jury-selection... https://www.cbsnews.com/news/donald-trump-new-york-court-cri...

Yup, the Meta hosted system is much more than LLaMA 3. Seems to have RAG, search, and/or tool usage