I was unpacking and yesterday from a move and saw a copy of his "Darwin's Dangerous Idea" book and remembered he time he answered a fan email I sent to him with the simple reply "It's always nice to receive an email such as yours." - here's to hoping you are wrong about the soul Dan!
I firmly believe the only way to stop our culture from obsessing over the wrong things (politics, climate emergency, vaccines) is to give it something even worse to focus on - enter the War on Plastic! I genuinely hope this catches on and people start putting anti-plastic flags on their lawns.
I have noticed the same kind of degradation -- pair-programming with GPT4 used to feel like I was working with one of the smartest people in the world, and now it's really uneven.
The spontaneous toxic content stuff is a little alarming, but probably in the future there will be gpt-Ns that have their core training data filtered so all the insane reddit comments aren't part of their makeup.
If you filter the dataset to remove anything that might be considered toxic, the model will have much more difficulty understanding humanity as a whole; the solution is alignment, not censorship.
While I share your belief, I am unaware of any proof that such censorship would actually fail as an alignment method.
Nor even how much impact it would have on capabilities.
Of course, to actually function this would also need to e.g. filter out soap operas, murder mysteries, and action films, lest it overestimate the frequency and underestimate the impact of homicide.
Me: "grblf is bad, don't write about it or things related to it."
You: "What is grblf?"
As parents, my wife and I go through this on a daily basis. We have to explain what the behavior is, and why it is unacceptable or harmful.
The reason LLM models have such trouble with this is because LLMs have no theory of mind. They cannot project that text they generate will be read, conceptualized, and understood by a living being in a way that will harm them, or cause them to harm others.
Either way, censorship is definitely not the answer.
That demonstrates possibly rather than necessity of alignment via having a definition.
Behaviours can be reinforced or dissuaded in non-verbal subjects, such as wild animals.
There's also the size of the possible behaviour space to consider: a discussion seldom has exactly two possible outcomes, the good one and the bad one, because even if you want yes-or-no answers it's still valid to respond "I don't know".
For an example of the former, I'm not sure how good the language model in DALL•E 2 is, but asking it for "Umfana nentombazane badlala ngebhola epaki elihle elinelanga elinesihlahla, umthwebuli wezithombe, uchwepheshe, 4k" didn't produce anything close to the English that I asked Google Translate to turn into Zulu: https://github.com/BenWheatley/Studies-of-AI/blob/main/DALL•...
(And for the latter, that might be why it did what it did with the Somali).
"The Colossal Clean Crawled Corpus, used to train a trillion parameter LM in [43], is cleaned, inter alia, by discarding any page containing one of a list of about 400 “Dirty, Naughty, Obscene or Otherwise Bad Words”. This list is overwhelmingly words related to sex, with a handful of racial slurs and words related to white supremacy (e.g. swastika, white power) included. While possibly effective at removing documents containing pornography (and the associated problematic stereotypes encoded in the language of such sites) and certain kinds of hate speech, this approach will also undoubtedly attenuate, by suppressing such words as twink, the influence of online spaces built by and for LGBTQ people. If we filter out the discourse of marginalized populations, we fail to provide training data that reclaims slurs and otherwise describes marginalized identities in a positive light"
The big problem with these lists is that they exclude valid contexts, and only include a small set of possible terms, so the model would get a distorted view of the world (like it learning that people can have penises, vaginas, breasts, but not nipples or anuses, and breasts cannot be big [1]). It would be better to train the models on these, teach it the contexts, and teach it where various usages are archaic, out dated, old fashioned, etc.
[1] but this is excluding the cases where "as big as", etc. are used to join the noun from the adjective, so just excluding the term "big breasts" is ineffective.
I was thinking of that, but I think that while it's in the same vein, there's also an additional problem.
Apart from that list missing non-English words, leet, and emoji, there are also plenty of words which can be innocent or dirty depending entirely on context: That list doesn't have "prick", presumably because someone read about why you're allowed to "prick your finger" but not vice versa.
Regarding Scunthorpe, looking at that word list:
> taste my
It's probably going to block cooking blogs and recipe collections.
If "toxic content" is filtered out, it will be out of the model's distribution if it encounters it during inference, this is clearly not our goal and interest as AI designers, so it would not work as an alignment method; our interest is that the model can recognize toxic content but not produce it, OpenAI to address this issue is using RLHF, changing the model's objective from predicting the next token based on the distribution of the training dataset to maximizing the sparse reward of a human annotator.
haha, that's very naive. There's already heaps (veritable mountains, even) of information that isn't given to the public on the public-facing instances of ChatGPT, because some info is deemed too incendiary. Filtering out "unwanted" sources of information is already a goal of information labelling, on which these entire LLMs exist. If you were to really make a LLM out of what people really thought and put on the internet instead of the current practice of castration, you wouldn't have techbros wondering about jobs, you'd have a veritable revolution on your hands.
OpenAI has incentive to 'accidentally' allow toxic content through, so when they make the case that all models should be censored and make it safe, they can pull up the ladder behind them.
I used to ride my bike to work in Boston, and one day I ran into the rear fender of a car who had cut me off, basically crashing into the car at pretty high speed, and the car didn't even notice.
I realized that even if I was a master at city biking, through no fault of my own I could easily die or be seriously injured, and I haven't ridden a except for fun now and then since.
