Do you even science, bro?
Hi! Can here. Today, we talk science.
I love science. Not in the I Fucking Love Science way but rather the scientific process is fascinating. In the collective imagination, science is about uncovering some profound truths. In reality, however, it is really more about reducing uncertainty to a degree where we can say "well, maybe my theory is not that wrong." The scientific method, which we honed through the Enlightenment Period, is not really about "establishing" truth as it is about "increasing confidence." There's never really any truth, just some shades of it, and only in as much as we can observe it. It's beautiful because as someone who has problems with ambiguity, I find it soothing to know that maybe it's OK not to know everything. I fucking love the scientific method.
Well, OK, I also probably love science, because I am not a scientist. I have a degree in science, and I spent four years immersed in a lot of science one way or another, but I never practiced it. The theory of science is beautiful, but to practice it is a grim affair. In many fields, the scientific work has become less about advancing the state of human knowledge (or, again, reducing uncertainty) but in fact about trying to get a paper published in a reputable journal. As they call it, it’s a world where you publish or perish.
It is depressing in many ways. People suffer through years of painstaking research to get their papers through the torturous peer review process so that it can be published in one of the reputable few journals in their field. You don't have to be a scientist to see the obvious problems here. The aforementioned journals obviously provide a lot of value by curating papers but more importantly, they provide anonymous peer reviews (ideally couple) so that there's also a quality bar. Yet, it's hard to shake the feeling that in an age where the cost of information exchange and storage is practically nil, we can probably do better than a bunch of publications not just gatekeeping the information on the supply side, but also keeping it away from the demand.
Anyway, obviously this, getting rid of the gatekeepers, isn't a particularly new idea. Cornell’s arXiv is the best known "pre-pub" (or pre-print) site where people can submit their papers after a much lighter review process. Papers are still "reviewed," but it's akin to a spam filter, rather than a full-blown peer-review process. As long as it's a "scientific" looking paper, with a couple known institutions and/or people attached to it, you can publish on arXiv. There are of course problems with this approach.
I am probably out of my depth here. But arXiv is more of a repository of papers than a curation. For many fields, the compromise between "full review" and "full access" tilts towards access for a good reason; the stakes are low enough, and the data and the methodology easily replicated (think computer science versus social sciences). There's a bit of counter-culture there too, but again, probably not the best person to appreciate the nuances here. I can only point to those who are.
Anyway, so a couple days ago, a new "paper" started making the rounds on Twitter where someone published a paper about how YouTube is actually a reducing radicalization. I say "actually" because the first author of the paper (again, this ordering of the name matters) not just published the paper on arXiv but then went on to Twitter to take explicit shots at Kevin Roose's investigation on YouTube on The New York Times.
Of course, it didn't take too long before someone —say, an actual scientist— actually read the paper, and pointed to its many glaring issues. For starters, the paper only looked recommendations with logged out accounts, while the NYT investigation was for logged in users. And not only that, the paper's research only looked at the recent algorithms, which had long been "revised" to reduce such recommendations. It is the kind of stuff that any researcher or anyone with a modicum of knowledge of the process would be able to point out if they put in the time, instead of hitting the RT button with fury. But, of course, by then, the tech Twitter, somewhat unsurprisingly, took to the paper as if it was a victory lap of tech over the media. The scientists had spoken, they thought, the media was wrong. It was...not great.
Look. I don't know what to say here, other than this stuff makes me sad.
It makes me sad because as someone who has been in tech around a decade now, where we went from being the weirdos to omni-fucking-everything gods, we still cannot bear what is really kids' gloves treatment from journalists. I will caveat that I am probably more steeped in that field than most tech workers, and do have a bunch of friends in the field, but I also do happen to know what happens when journalistic accountability ceases to exist. There's a lot I have to say here, but I'll spare you the TED talk.
But there's another point here about how so many people do not actually know how science works. Again, in the romanticized version of science, people write papers, and every so often, one of them changes the world. Of course, that does happen, for example, if you discover the double helix structure of the DNA, or, I don't know, the theory of relativity. But even then, the story is more complicated.
Generally, the real currency of scientific advancement is not a single paper that gets cited a lot, but instead a growing body of evidence. We do not, for example, generally cite a single paper for calling out that climate change exists or that it's caused by human activity, but the general consensus of the scientific community which keeps on publishing research after research on the topic. You need not just a body of evidence, but also it needs to be growing. It is only at this point where we claim, somewhat misleadingly, that something is "established as a scientific fact."
The parallels between how the public's (and the big chunk of media's) illiteracy of how scientific knowledge accumulates and the Fake News Wars of 2015-2019 are too on the nose to pass up. Internet destroyed, as the saying goes, the local monopolies of newspapers. Everyone now is a publisher, which means no one really is. We have neither the gatekeepers nor the protectors. And just like that now, the scientific literature is no longer bound to a few journals, which is good, but now all papers look alike regardless of their source, which is bad. Just like we don't know whether we read came from a reputable news source or some kids in Macedonia, we, both the public and the media, don't know how to judge a scientific paper's veracity.
In the end, I don't have a good answer to all this. When I first tried to make an outline for this piece, there were so many things I wanted to write about; like how well-intentioned (or not, in this case) researchers might be to do such investigations, the black-box nature of such algorithms make it impossible to really do any research from the outside. Or how trying to extract any data to be shared with researchers is periled with its own various privacy challenges, that I am not sure can be overcome.
I wish I had ended the year on a happier note.
Happy New Year, everyone.
OK, I wrote this last weekend. And I don’t think it’s fair to end what was a great year for me on a negative note.
I had entered 2019 while on a plane from San Francisco to Paris. Let me assure you, it was less of a globetrotting elite feeling than it was a depressing and lonely one. 2019 turned out to be a year where I had moved around quite a bit (for…reasons) and settled back in San Francisco.
I have found a newfound passion in health and fitness (much to my Twitter followers’ chagrin), and in the best shape of my life, only hoping to get better, as long as I stop injuring myself doing household chores. I can’t tell you how life changing it is to be able to go from being a couch potato to running 10Ks on a whim.
Also, I have found a job that I couldn’t be more excited about, partly through this newsletter that started out as more or less a joke. As I thought through all the problems created by the information businesses in writing, I stumbled upon a bunch of folks who have the same concerns but also have plans to fix them.
I am looking forward to what 2020 brings. Thanks for being dear readers and hope to see you again soon!