It is unclear this model should be on that leaderboard because we don't know whether it has been trained on mteb test data.
It is worth noting that their own published material [0] does not entail any score from any dataset from the mteb benchmark.
This may sound nit picky, but considering transformers' parroting capabilities, having seen test data during training should be expected to completely invalidate those scores.
Though not relevant to the Atlantic ocean, it is worth noting that the Australian coral reef has in fact recovered from the disastrous state it was after the bleaching and damage events in 2016/2017. Reef coverage is in fact at the highest levels ever recorded in some areas.
It does make one wonder why this more recent good news has not found the same level of exposure as when the situation was bleak back then. Perhaps good news just doesn't gather as many clicks.
I believed the author of the blog was referring to tech roles specifically (not sales or similar).
I can say from personal experience that, at least in Australia, tech workers for mining companies that work in city offices are paid fairly similarly to other non-FAANG tech workers (e.g. banks etc). I also just checked levels.fyi for a few big mining companies and verified that this is the case.
Engineers (mining, geo, tech, whatever) that work out in the field do make quite a bit more. MAYBE around what FAANG would pay, but FAANG still pays more after a few years of refreshers and/if one manages to climb the ladder.
I think the author is quite off with their estimates of what people make in "heavy industries". At least as far as my experience goes for AU. FAANG/HFT still beats everything, and by a vast margin.
Though I agree with the idea that MLPs are theoretically more "capable" than transformers, I think seeing them just as a parameter reduction technique is also excessively reductive.
Many have tried to build deep and large MLPs for a long time, but at some point adding more parameters wouldn't increase models' performance.
In contrast, transformers became so popular because their modelling power just kept scaling with more and more data and more and more parameters. It seems like the 'restriction' imposed on transformaters (the attention structure) is a verg good functional form for modelling language (and, more and more, some tasks in vision and audio).
They did not become popular because they were modest with respect to the parameters used.
The reason why people care about privacy is not necessarily because giving up privacy has some directly observable negative effect. But, simply, living without freedom sucks.
I don't want you to know my personal information not because you could/would do something nefarious with it. I don't want you to have it simply because it's none of your business.
I am personally starting to think that this is the framing we need for this conversation. It needs to be discussed as a freedom as opposed to a right for no other reason than 'expectation of privacy' was very successfully neutered.
It's the first mobile app I have ever written and I enjoyed the process quite a bit!
My main goal was to deliver better identification accuracy than similar apps.
However I also wanted to provide useful plant information along with the identification and naively thought that this would have had to be a solved problem - surely there would be some online DB with all plants data neatly organized (I'd be even happy to pay for it!), in particular plant care information - but alas!
Lack of differentiability is actually a very important feature of the underlying optimization problem.
You might think that it doesn't matter because ReLU is, e.g., non-differentiable "only at one point".
Gradient based methods (what you find in pytorch) generally rely on the idea that gradients should taper to 0 in the proximity of a local optimum. This is not the case for non-differentiable functions, and in fact gradients can be made to be arbitrarily large even very close to the optimum.
As you may imagine, it is not hard to construct examples where simple gradient methods that do not properly take these facts into account fail to converge. These examples are not exotic.
Only tangentially related, but I have been trying to enroll for Apple's developer program for almost 3 months now.
Understanding what the problem is is essentially impossible. Going to a physical store doesn't help, calling their customer service has them telling you to go to www.apple.com/support (???), and writing for support has them rotate you through 4 different, and decreasingly useful, representatives.
The last response I got I was told the issue had to be handled by yet a different representative and it would take an "indefinite amount of time". Which may be a nice way of them saying it's never going to happen.
It really is demoralizing when you realize there is nothing you can do really, even in cases when you have done nothing wrong.
A friend and I spent a month or so building an iOS app we were hoping to release and monetize, but we're also entirely unable to get a developer account created. Corporate entity, DUNS number, American, extremely boring people, and just a generic "Error creating developer account" on the signup form. Apple's support was hopeless in helping.
We gave up and re-built it as a web app. The thing that convinced me was the realization: When was the last time you installed/used a non-game App on the app store that, by your assessment, has less than 1 million users? I looked down my list of installed apps and realized that indie apps are kinda dead anyway. And our web app has been pretty successful.
Just curious, with the web app, how has the experience been for your Apple users vis a vis the Androids? Are you seeing some reduction in expected footfall because of your web app decision?
I had similar issues, and I wish I could remember what solved it. It was something stupidly dumb like I had to log out and log back in on my phone or something. There have a couple of different edge case bugs that prevent people from signing up, and Apple customer support is useless on this.
Same here. It was something trivial with the form that I fussed around with until it worked, or maybe I didn't have iCloud enabled at all and the form didn't alert me about it.
I've had a similar problem trying to renew my Apple developer account. Had it for over 10 years. I had an email a few weeks ago telling me it could not automatically renew (same bank details that worked fine last year). Nothing I could do on their website would make it work. I got hold of someone on their online chat who directed me to the Apple developer forums.
I gave up in the end. But I will have to sort it out before I can release the Mac version of my current project.
That's a funny take. I guess Apple is going to pay my sick leave, then? Buy me the hardware I need to do my "work for them"? No? Weird, guess I'm not working for them at all in any way.
No, you're right, it's actually worse than if you worked for them. Lmao. Really the worst of all worlds. You're dead in the water with out their platform, without their grace, or with all of those things, but their incompetent auth platform.
You could reframe that easily by saying that without Apple making the hardware and services exist, there would be nothing to run your app on. It’s a symbiotic relationship: devs need Apple and Apple needs devs.
I'm not sure what your point is, but I 100% agree with you. Apple is awful, and you have to be downright masochistic to develop for their platforms. Thinking you're their employee when you develop for their platform is laughable.
This requires a Dun & Bradstreet DUNS ID number, which isn’t the most difficult thing in the world to obtain, but also isn’t trivial, especially if you don’t actually have any formal business documents.
Yeah, can say from recent experience this just adds _more_ steps and opportunities to ghost for a couple weeks, get another vague email, ghost for a couple weeks...took me about 3 months to get it all going.
The DUNS stuff was pretty funny. All flows related to getting an ID have a big "Are you doing Apple dev stuff?" button. It's like Apple outsourced support to them. Apple's DUNS lookup tool saw my business and the correct DUNS number, but trying to register with it got an error...eventually dissipated after a couple weeks. Same story for registering an account in the first place: it refused to register james@tld.com, where tld is a Google Workspace account, with no discernable error. Again, dissipated after 3 weeks, thankfully.
Would you be able to elaborate on your approach to TDD with spark sessions? You can persist them, which is only useful if you are doing multiple tests in a run.
But I find myself running one given test, making some code changes, and then wanting to run it again, over and over. Instantiating a local spark session takes several seconds every iteration. Enough for me to often want to "alt tab" into something else instead of waiting. It's very disruptive.
I did not know about Fugue but will definitely give it a try. Looks almost too good to be true.
It is worth noting that their own published material [0] does not entail any score from any dataset from the mteb benchmark.
This may sound nit picky, but considering transformers' parroting capabilities, having seen test data during training should be expected to completely invalidate those scores.
[0] see excel spreadsheet linked here https://blog.voyageai.com/2024/09/18/voyage-3/