> So, are reasonable examples now of these models allowing semantic context?
This is about where I am stuck. I'll start believing that we truly are on the cusp of a revolution as soon as I see Google Translate reliably knowing when to translate "home" into French as "domicile", "foyer", something those lines, or as "accueil."
Right now it seems to very frequently choose "accueil", which is generally wrong, except when you're talking about websites and software user interfaces. That it's biased so strongly toward that error speaks volumes about how critical semantics are to sorting out natural language, and also about how bad current NLP systems are at dealing with semantics.
Syntax and semantics were developed for human language, yet it's much easier to puzzle the difference in a computer language than in human language. With syntax and semantics so wrapped together, however, it kind of seems like you can go a long way with just capturing syntax, rhythm, word choice and etc. Which is to say the semantic side can be even worse than it seems, ie, nonexistent.
Not offhand. It's a problem I've noticed more often when looking at translations of larger bodies of text, though. Translating simple, highly standardized sentences like those is what neural translation does best, because it is able to basically just consult an internal phrasebook that it's compiled from its training data.
> This is about where I am stuck. I'll start believing that we truly are on the cusp of a revolution as soon as I see Google Translate reliably knowing when to translate "home" into French as "domicile", "foyer", something those lines, or as "accueil"
Isn't that basically the same as the Winograd problem?
I would guess that it's a bit easier. With what I was proposing, you just need to be able to infer the semantics of certain words. With the Winograd problem, you need to grasp the semantics of all the words, and then use that knowledge to infer a pronoun's antecedent based on what yields a more sensical overall interpretation of the sentence.
"use that knowledge to infer a pronoun's antecedent based on what yields a more sensical overall interpretation of the sentence"
People often assume a very benign, civilized environment for AI. In ordinary life, human beings (on the internet or off) take an adversarial approach to other people modeling the logical framework behind an utterance. They either subvert it for amusement (trolling or comedy) or profit (politics, propaganda, sales), and a tremendous amount of effort goes into it, much of which is very effective.
As people have observed, it can be very easy to convince a human that a machine is intelligent, as with ELIZA. But if you violate the presumptions of trust that a machine is designed with, it's going to be surprisingly vulnerable. To be superior to humans, a machine would have to be able to fend off an intelligent person trying to undermine it, not just work when it is spoon-fed.
This is about where I am stuck. I'll start believing that we truly are on the cusp of a revolution as soon as I see Google Translate reliably knowing when to translate "home" into French as "domicile", "foyer", something those lines, or as "accueil."
Right now it seems to very frequently choose "accueil", which is generally wrong, except when you're talking about websites and software user interfaces. That it's biased so strongly toward that error speaks volumes about how critical semantics are to sorting out natural language, and also about how bad current NLP systems are at dealing with semantics.