PubPub folk here - our landing page is pretty much 6 months past its shelf life. A rework that helps people find relevant communities is in the pipeline. In the meantime - we have a simple Explore page[1]. Some of my favorite communities are the newly launched Harvard Data Science Review[2], The Journal of Design and Science[3], _Cursor[4], and the Stanford Journal of Blockchain Law and Policy[5].
We're also encouraged by a handful of communities that are experimenting with new formats of pre-publication review and training such as: Collective Wisdom[6], EFPT Psychotherapy Guidebook[7], and Data Feminism[8].
At a more meta-level, the challenge here is that this ecosystem is overwhelmingly complex. The business-models, culture, and technical infrastructure are all unhealthy and self-reinforcing. There's no single platform that can fix this - it needs a political campaign in parallel with a social movement in parallel with some really good tech infrastructure. The US healthcare challenge is a good parallel in a lot of ways. Both are going to need new economic and governance models. At the risk of straying too far from the original question, PubPub's parent - the Knowledge Futures Group[9] - is working to push on all of these fronts. Feel free to reach out if this is the kind of thing you care about!
PubPub | Full-Stack Web Developer | Cambridge, MA | Full-time Onsite | https://www.pubpub.org
We're building an open-source publishing platform for collaborative scientific (and other) documents.
Started as an MIT Media Lab project, PubPub is being put into production pipelines in close collaboration with the MIT Press. You'll work from the MIT Press and MIT Media Lab offices with the current team of 3-5. As a small team, you will play a critical role in the design and development of the product going forward.
Requirements: Javascript, React, Node, and open source community management experience. 3+ years web dev experience. Frontend design experience (Sketch, or similar).
Great question, and one that certainly doesn't have a straightforward or trivial answer. It's definitely more of a social challenge than a technical one - making publishing free/open won't do anything to fix incentives on its own.
My hunch is that change to this system will come from the outside. It's too risky of a career decision for a tenure-track professor to start publishing on PubPub (or any open/new system). But, there are lots of people who aren't playing that game. Lots of people who are doing science outside of academia, at a corporate R&D position, or for the sake of education, etc.
The most important step is to show that open publishing works. If we can work with these early adopters and show that conversations are more rich, or results more reproducible, we can start to go to universities and grant agencies and advocate for them to require open publishing. The first day that a university hires a professor or an agency rewards a grant based on the history of openly published work, will be a turning point. I hope it will be similar to the first time a software dev was hired for their Github profile, rather than their CS degree.
Today, software companies hire on experience. A university degree can show that, but so can major contributions to an open-source project. I hope science can become the same. Whether you're a PhD out of a great program, or an high-school drop out that has committed her life to rigorous experimentation, your demonstrated experience should be what you're hired on, not the list of journals that have found it in their interest (many of them are for-profit) to include your work.
Perhaps offering some sort of crowdsourced funding mechanism and a reputation system would go a long way toward correcting some of these incentives?
For example, giving authors / organizations a Bitcoin address where they can receive funds from individuals / organizations who want to support their research.
Also, awarding reputation to authors based on the level of peer review their research has successfully undergone (number of peers, level of rigor, etc.), and conversely awarding reputation and funding to those who perform peer reviews. Allowing users to contribute to a peer review fund for individual articles or in general.
All that to say this is very exciting and opens up a lot of new possibilities.
> For example, giving authors / organizations a Bitcoin address where they can receive funds from individuals / organizations who want to support their research.
That's a fantastic idea. Maybe we could call this "depository" of money to conduct research something like, hmmm, what's a good word… a grant?
> Also, awarding reputation to authors based on the level of peer review their research has successfully undergone (number of peers, level of rigor, etc.), and conversely awarding reputation and funding to those who perform peer reviews.
Sounds fantastic as well! Maybe these authors could create like, a website or curriculum vitae where they could list their accomplishment to establish their reputation. You know, they could have a section in their medium of choice that could be titled something like selected peer reviewed articles where they'll list their publications along with their coauthors and the journal it appeared in. Maybe these journals could devise some kind of ranking to measure reputation. Maybe they could call it something like… amount of impact or maybe just impact factor for short. I think this could work really well.
> Allowing users to contribute to a peer review fund for individual articles or in general.
Maybe a general fund should be created to support science! Maybe a national science fund or something, governed by a so-called national science foundation who can vote scientists, engineers, and the like onto their board to steer the allocation of funding.
I really think you're onto something very good here!
> Maybe we could call this "depository" of money to conduct research something like, hmmm, what's a good word… a grant?
Nah, that word is already in use for stagnant allocations of academic welfare to work on bullshit instead of transformative techniques (e.g. CAR T-cells, which NIH refused to fund for years). Need a new word to signify "money that is actually intended to produce results" instead of "a pension for irrelevant tenured senior PIs to pay non-English-speakers below-minimum-wage to work on topics that became irrelevant a decade ago".
> Maybe they could call it something like… amount of impact or maybe just impact factor for short. I think this could work really well.
Ah yes, impact factor is such an amazing tool. It allows "executive" "leadership" types to predict (very poorly, but who cares?) how many citations a paper might receive if it survives the months or years between submission and publication in a major journal. Trouble is, JIF is massively massaged and the COI that Thompson Reuters has in equitably enforcing it is ridiculous.
WARNING: Non-peer-reviewed work ahead! If you're not careful, you might have to apply critical thinking to it!
> Maybe a general fund should be created to support science!
That's a great theory. Perhaps it can be as well executed as the CIHR fund (where study section has given way to "ignore everyone who doesn't suck my dick directly") or NSF (whose yearly funding is dwarfed by the R&D funding at a single company). This approach is working out very well!
You know, if I didn't know better, I might think you were the sort of researcher that fails to look at the details and just submits your most fashionable bullshit to whateve journal at which your pals happen to be editors. I might get the impression that you're the cancer which is killing grant-funded science, which prizes large labs over large numbers of R01 projects, which believes that O&A is an entitlement to take out mortgages on new buildings instead of to pay for the costs of disbursing and administering a grant. But, since the evidence isn't before me, I won't.
It would be nice if you thought a little more carefully about what you wrote. The devil is in the details.
> or NSF (whose yearly funding is dwarfed by the R&D funding at a single company)
If the worst thing you can say about the NSF is that they need more money, that makes it sound like GP has come up with a nice way to allocate the available funding towards particular research projects.
> It would be nice if you thought a little more carefully about what you wrote. The devil is in the details.
Details like how to get "crowdfunding" to put up enough money that "independent scientist" can be a full time job and not just a hobby for the odd few who somehow already have most of the needed lab facilities/equipment?
Also: I still haven't heard (from either you or the previous parent poster) how journal impact factor can possibly be justifiable as a metric for relevance.
Anyone surveying the actual citation distributions at major journals will immediately note that a metric assuming near-normality cannot possibly summarize non-normal distributions of citations. The latter describes nearly all journals, thus even if JIF were not manipulable by stacking, self-citation, and negotiated exclusion of items to decrease the denominator, it would still suck.
Look carefully at the details! This metric is among the most frequently emphasized by researchers who comprise study sections, and it is objectively terrible.
I'm not whining "just because" -- many of the lines in my CV end with NEJM, Nature, or Cell (no Science paper yet). I'm saying that at least one of the commonly accepted metrics for individual investigators is broken. That sort of detail corrupts the entire rest of the system.
I'm also not saying that a direct public-facing system wouldn't have huge potential problems (although it is nice to see attempts like experiment.com seemingly doing OK, and the funders realizing, hey, there are a lot of shades of gray between "utter bullshit" and "exactly the right experimental design for the question being asked").
One of the nice things about talking directly with folks at NIH, for example, is that they recognize there are serious issues with the incentives in place. If they are willing to collect the data and evaluate (publicly, e.g. in Chalk Talk postings) the findings, doesn't that suggest room for the current system to improve?
I take it you're not familiar with "crowdfunding" sources like the AACR, LLS, ASCO, or other professional societies?
As someone who is funded by several of the above, and who noted that their review processes were substantially less bullshit-intensive yet no less rigorous than NIH review (which has many benefits, efficiency not among them), I'm going to go out on a limb and suggest that it's possible.
As far as the NSF, they do a good job with what they have, but what they have is not commensurate with what we as a society could stand to spent on science. Even NCI is a far cry from that: https://pbs.twimg.com/media/CmLJzKQWkAAl372.jpg:small
Distributions are similar for various other avenues of funding, and it is quite clear that the overhead & administrative costs requested by many recipient instutions are far out of proportion to actual needs, so the impact of the funding allocations is further reduced.
Thus it appears that a direct conduit from potential patrons to researchers is, in fact, desirable. Otherwise, services like experiment.com would not exist. They're not at the level of an NIH study section (duh?) but they have consistently produced a small stream of usable results that belie their supposed irrelevance. Once upon a time, the Royal Society existed for just such matchmaking: find a rich patron and a promising young scientist and line them up. You've likely noticed that many if not most major universities and "centers of excellence" rely upon exactly this model, supplemented with NIH or NSF grants, to exist. Further modularizing the model so that an administrative hand yanking out bloated "indirects" at every turn is not mandated might not be the worst thing, or (alternatively) being more transparent with said O&A requests, might at least bring some of the bullshit under control.
The public clearly wants accountability. The masses may be asses, but if we want their money, we really ought to be transparent about what we're doing with it.
The difference between professional societies and crowdfunding is that professionals, not the crowd who donate directly, decides which projects to fund. In this sense, I do not see a great qualitative different to government funding agencies --- if you do, please elaborate.
EDIT: And to clarify, in the societies I know, general members do not directly take part in grant decision processes. Rather, the decisions are made by a small panel, possibly together with external reviewers. This is fairly different from crowdsourcing.
It's different from crowdsourcing, but the source and sink for the funds also tend to be more closely related. Ultimately I don't really believe that major initiatives (eg P01-level grants) can be adequately reviewed by anything other than genuine peers.
But by the same token, an exploratory study requesting $30k for field work or sample processing could very well be evaluated by less skilled peers. Actually, I think I'm going to try and shop this to a friend at NIH. I'll fail, most likely, but at least I won't just be whining.
For example, pharma and big donors use the LLS review system as a "study section lite" to hand out grants larger than a typical R01. The paperwork and BS isn't really necessary at that level and just gets in the way. If something like this existed for "lark" projects, inside or outside of NIH/NSF, perhaps more diverse and potentially diversifying proposals would be worth submitting.
To some (fairly large, in the case of ASCO or ASH or AACR, perhaps smaller for LLS or AHA) degree, the dues-paying professionals in these societies are the crowd. I would say they are a middle ground between something like an experiment.com or similar at one extreme, and NIH (which has inordinate purely political input -- ask your program officer!) at the other.
We shan't discuss scams like Komen here, but genuine research foundations can exist along a continuum.
The paperwork burden for an NIH grant (relative to a society grant) is often a large scalar multiple. The accountability is often on a par with, or less than, the typical society grant. It mystifies me why this should be so.
I feel like those fields with highest facility needs / costs would come last if at all. There are many fields that require pretty small amounts of resources for example: Computer Science (I did most of my research on a personal laptop, with other equipment costs <$5,000), Mathematics, Philosophy, Economics, Psychology.
All of these seem very possible to crowdfund with the ultimate goal of unhooking them from perverse incentive systems of typical universities.
at least three of the above are in fact supported by experiment.com backers, although largely as a "bridge" to more traditional scholarly outlets. That said, if you go out and get extramural funding for your work, generally that is the definining characteristic of a successful PI, so...
I wasn't saying the current system is great, there's a lot wrong with it yes I agree. The impact factor thing is also a pretty silly metric to me as well, I agree with you there. The point I was trying to make, albeit sarcastically, was that the system the guy proposed is what we have today just without the extra hoops to jump through. Like I have absolutely no interest in maintaining a Bitcoin wallet or whatever nor do I want anything to do with them. I'll take my funding in dollars or euros or something real and tangible please.
Mostly agreed, although I did consider using an HPC allocation to mine bitcoins & hire work study students. But then it turned out that if you study interesting stuff and write the ad correctly, you'll have to beat them away with a stick. For good measure, I convinced one of our corporate patrons that they ought to pay for one of the students.
As far as extra hoops, it's not clear to me whether endless NIH paperwork bloat and ICMJE declarations are more or less onerous than crowdsourcing type stuff. I tend to think there must be a happy medium, but I could just be naive.
Oooo sarcasm. You're probably right though, the old system seems to be working out pretty well. Besides, science is all about never questioning existing institutions right?
We're keen on getting good offline support added, but (as you point out) there's still a lot to be done on the web side of things. We're pushing out some big updates that hopefully clarify the project and how to use it in a couple weeks.
Welcome to ideas on what good offline support would look like, or PRs if anyone wants to take a stab at it.
Our goal is for PubPub to be a public utility for scientific communication. It'll be non-profit and open source for as long as it lives (still a grad student project at the moment...) and free for anyone to publish whatever they like. I don't think we're nearly smart enough to know exactly what that looks like, so please do feel free to contribute ideas, code, or inspiration into what a public tool for science communication should be (this comments page is already wonderful in that regard).
PubPub here. We need cleaner documentation on this - apologies for the lack of clarity.
PubPub journals (like JoDS) are no longer the publisher, but rather the curator. So, the key steps are:
1) Anybody can write and publish a document on PubPub.
2) Any document can be submitted to any journal.
3) Any journal can choose to feature a document (regardless of whether it was submitted or not).
4) Documents can be featured in unlimited journals.
So, to answer your question: yes - anybody can (publish on PubPub and then) submit to JoDS.
Thanks for the idea! We've gotten a bunch of feedback asking about more specific categorizations once the basic set of votes are saturated. This could be a pretty fun way of getting at something like that. We'll keep it in mind going forward!
Doh' - a check showed that we were "load balancing" across all of 1 port. Pulled from the wrong git branch on one of our updates apparently -_-. Running through many more node instances now - hope it's up for you. Thanks for pointing it out.
My encryption is to keep everything on a hard drive that isn't connected to the internet. The insurance companies would no doubt like to see my timeline on my health category with a timeline on dates of every issue I have ever had with the hundreds of sub components of my body with everything from acne to heart trouble to ringing in ears to joint pain.
I have a category with everything I intend to do to the child if I have kids. I have a category with pictures of me every year from 3 days old to today. I do the same with my parents and grandparents (which is more spotty).
I have a category on all the friends I've ever had, what they liked, why they liked me, what I did to become aquainted, birthdays, their parents, all the information about them you would expect a close friend to have.
I've got all my tax returns back to 2005 (I started the wiki somewhere in 2007). My current net worth plotted on a chart, documented by what I did which caused certain plunges and rises in that graph.
A big category on girlfriends, kinds of women and what works with some. I can give you a cruise through every significant romantic event in my life from my first kiss back in college to sexual encounters. With photos of each girl. Each girl I dated given a full psycho analysis, funny how I searched out each edge of the (Smart, Sane, Pretty pick any two) triangle. I even dated a few (1 of 3) and (0 of 3). Big mistakes that haunt you.
I have a category of books I've read, and plotted those against my life timeline. What I've thought of them, how they changed me.
Anyone who got ahold of this wikipedia and had 20 hours to read it all would know as much about me as I know about me.
This wikipedia is what I want to see on dating websites, If I could include this on a dating website, and have all the girls do the same, it would make dating a lot easier. I imagine a post with dental records, tax returns, employment, income histories, aspirations, books read, history of diseases/conditions, family genetic lines, computers owned, friend structures, self psychoanalysis, and hundreds of other subjects which makes us unique from other humans.
A big worry is that the hard drive bricks iself or gets stolen, this wikipedia is worth tens of thousands of dollars to me. My backup strategy is to make a hard drive cloned image every 3 months with clonezilla, with the hd stored in a physically different location. so my wiki would even survive an fbi takedown where they steal all your stuff.
I've got all the programs I've ever written saved, so I can go back in time 4 years and see programs I wrote in C++, Visual Basic. It's like reviewing your essays from first grade, I can't bear to see them, my programming skill has gone up slowly but surly over time.
Wow, that's really amazing. I think I'd like to start doing the same kind of thing. What exactly to you use to accomplish all of this? What wiki? Plugins? Anything else? Do you have any other advice for someone wanting to get into this? Or something you would have liked to do differently from the start? Any advice you have would be wonderful.
"bluefish editor" freeware on top of a linux operating system (Fedora core). I suggest a more user friendly version of linux for non programmers. I've made short shell scripts which copy templates to different keyboard combinations. I have scripts which validate the html I've written. One of the things that bogs you down is getting image link text from camera/scanner into the html file, so I automated that as much as possible so It is just a matter of opening the scanner and hitting one hotkey which kicks off a script, and scans the image, puts in in the images folder (prompting me for a filename) saves it and injects the html into my bluefish editor. Same thing for photographs and videos on my camera. It's all gotta be (One click-add) or it will take hours each session just fiddling around with documents.
The real secret is adhering to the rules of a wikipedia. Nothing may be added without establishing its link and relevancy to another page. Also, Here is the secret sauce, a tough rule is that when you introduce a new page that links to an old subject, which introduces new information, or new evidence which contradicts old pages, you have to spend time and re-build the pages being linked to. I have a system where editing one significant page has the possibility of having me go back and change previous observations, the benefit is that it automates your thinking process. If you keep all your thoughts you ever have into a tight tree form, you can find that your human memory is upgraded a hundred times over. I was able to use process of elimination to determine which foods were causing my acne, and through continued graphing and elimination, was able to find what elements in which foods were causing it. Something many doctors would probably like to see. When I go to a new dentist, I sometimes bring along my entire history of each tooth, each operation i've had, i can see where that time I kissed the wrong girl caused a microbe party in my mouth which caused cavities. Though the dentists have no idea what microbes are in my mouth, through my wiki. I have a good idea. In some ways the wiki makes me a better dentist (to myself) than the professionals.
It's a wikipedia that treats myself like scientists would treat a newly discovered intelligent alien craft. Everything about it is described, graphed, analyzed, compared, and charted. Nothing about it is taken as a given. The part that keeps me coming back to it is my thirst for knowledge. When I browse hacker news, stackoverflow, and (years ago, digg), anything that struck me as useful for growing myself or growing knowledge was included. So now I have like 800 items in a list organized by category and awesomeness, that to this day when I look at it, I still see the importance. I have a category of guitar songs I can play, piano tunes, how I learned. etc.
Whenever I hear a song that provides a certain response, sadness, happiness, euphoria, depression. I write it down on a receipt, then when I get home I log it. So I have a series of songs I can play which seriously induce all these emotions. I try to stick to the happiness songs. there is a boat load of ones that cause the other emotions. As a result I have 30 songs that when I play, suddenly make me happier. Sometimes when I'm depressed I play the depression inducing songs and it makes me extremely depressed, and I can look at myself like a computer, an input output device, and I see that depression is only a response to external stimuli, if you can take control of your external environment, you can custom make your emotion for that hour/day.
We're also encouraged by a handful of communities that are experimenting with new formats of pre-publication review and training such as: Collective Wisdom[6], EFPT Psychotherapy Guidebook[7], and Data Feminism[8].
At a more meta-level, the challenge here is that this ecosystem is overwhelmingly complex. The business-models, culture, and technical infrastructure are all unhealthy and self-reinforcing. There's no single platform that can fix this - it needs a political campaign in parallel with a social movement in parallel with some really good tech infrastructure. The US healthcare challenge is a good parallel in a lot of ways. Both are going to need new economic and governance models. At the risk of straying too far from the original question, PubPub's parent - the Knowledge Futures Group[9] - is working to push on all of these fronts. Feel free to reach out if this is the kind of thing you care about!
[1] https://www.pubpub.org/explore
[2] https://hdsr.mitpress.mit.edu
[3] https://jods.mitpress.mit.edu
[4] https://cursor.pubpub.org
[5] https://stanford-jblp.pubpub.org
[6] https://wip.mitpress.mit.edu/collectivewisdom
[7] https://epg.pubpub.org
[8] https://bookbook.pubpub.org/data-feminism
[9] https://kfg.mit.edu