Monday, October 16, 2017

What works and what does not work in contemporary science?

Today I participated in a workshop on the way forward for taxonomy in Australasia, so it might be a good opportunity to come back to the topic of the bioinformatician who thinks that all of science is broken and to consider what works and what does not work, at least in my view, in the field of science that I have the most direct insights into. I am not limiting myself only to taxonomy but will include all the broader field of systematics, but taxonomy is a major component.

Funding: Everybody says that funding in their field is too low, so this applies across all of science. But are scientists just whining? No, I believe that there is indeed too little competitive research funding available.

First, I have seen and heard of lots of cases where funding agencies have to reject very valuable proposals because there simply isn't enough money to fund everything that would be good to fund (sometimes apparently called 'approved but not funded'). Second, there are many funding agencies where you have success rates on the order of 2-10%. So to conclude that funding levels for competitive grants are high enough we would have to believe that 90-98% of applications are useless and that the weeks that the unsuccessful applicants have each invested into writing their many applications could not have been used in a more productive way. And that seems like a big ask.

Incentive structure: This is the big one, at least to me, and I guess here I find the most overlap with the aforementioned frustrated bioinformatician. What basically happens is that people are rewarded with jobs and promotions for (a) having publications in JCR-listed journals, in particular if those publications are cited a lot by other papers in other such journals, and for (b) getting external research grants, but of course the decision whether somebody gets a grant is also partly and sometimes mostly based on criterion (a). This is simplifying a bit, as there are also, depending on the job, teaching, textbook writing, conference participation, etc., but not by much. Publication lists are usually the key factor.

The problem is not that publications are a key factor though, because if a scientist does not publish their research it is indeed wasted. The problem is that there are lots of useful outputs that scientists can produce that are not, very specifically, research papers in JCR-listed journals.

Perhaps the most impactful thing a taxonomist can do for end users is to produce a publicly accessible online identification key or to contribute to a flora. But no matter how often this output is used to identify organisms, how many people need it for their work, it does not count the tiniest blip towards the taxonomist's number of citations or their h-index. There is no requirement for the end-user to cite a key in a paper, even if they used it during their work; and even if people cited it, it wouldn't count because an online key or flora volume is not captured by the JCR. Consequently, in terms of career advancement the taxonomist would have wasted their time and should instead have produced journal articles cited by other journal articles.

It is clear that people largely do what is rewarded, and largely cease doing what is not rewarded. So to the degree that there are useful things for scientists to do that do not result in publications in JCR-listed journals the incentive structure in science leaves something to be desired.

More generally, I feel that there is too much of a focus on flashy results and innovative methods but too little appreciation of incremental, everyday work. One of the surer ways to be cited a lot appears to be to develop a new lab method or a new piece of analysis software. This visibly leads to conferences full of rising stars each promoting their own new Bayesian analysis method or bioinformatics pipeline, but very few early career researchers contributing to specimen identifications, describing new species, or conducting taxonomic revisions.

Now to publishing itself. Apart from what I wrote in the previous section, in terms of academic papers I am actually not all that unhappy with the situation. Yes, ideally one would have all journals run as public utilities, cost-free to publish in and cost-free to read, instead of having private quasi-monopolies with massive profit rates and, pick your poison, either research locked away behind paywalls or money that could go towards research spent on publication fees.

But in a system where somebody has to pay I prefer subscription-based funding instead of author-pays open access, which is promoted by many people frustrated with the status quo, because in the latter system the incentives are perverse: journals are financially rewarded by accepting as many papers as they can instead of maximising the quality of their content.

As for peer review, again the system as currently implemented seems to work reasonably well; that is why it evolved to be like it is in the first place! I have received good feedback in many cases. I also had one or two cases where I believe the manuscript was unjustly ripped apart by an individual reviewer, but well, there are human egos involved, and one should not make the perfect the enemy of the good. I am trying to be a charitable and constructive reviewer myself but also suggest rejection papers where the conclusions do not follow from the results or where the methodology cannot address the research question.

If there is anything that I see as a current problem it is that there are rumours of journals increasingly being unable to find enough reviewers, which suggests either a lot of free-loading going on or journals being too unimaginative with reviewer invitations, or both. (Certainly I do not appear to get as many invitations from mid-level plant systematics journals as I would expect if they are struggling to find referees.)

Reproducibility: As I wrote in the previous post on this issue, I do not see any evidence whatsoever that taxonomy, phylogenetics, systematics or evolutionary biology have a reproducibility problem.

So that is how I at least perceive that part of science that I can judge best, for what it is worth. More money would be good, but an even more intractable problem is that the incentive structure currently in place does not reward some of the most useful and impactful work that systematists could be doing. Note that neither of these problems would really be solved by scrapping journals and publishing everything on preprint servers, but more on this maybe in another post.

Everything is about white male privilege, even writing advice it seems

I read a headline saying Why the writing advice 'show, don't tell' is inherently political and thought, well, this should be good. The links ultimately lead to an essay called Let me tell you by one Cecilia Tan.

The author discusses 'show, don't tell' (SdT) entirely in the context of world building, i.e. info dumps about the background of a story. She then argues that SdT relies on a shared cultural background, and thus this writing advice privileges writers who can rely on sharing such a background with their readers, i.e. white males.

Now, first, I would not see anything particularly wrong with this in principle, because why should it only apply to white males? If an Iranian woman wrote a novel for Iranian women, it would work the same.

But more importantly, at least to me, and while I appreciate that I am not an author of novels who has run into that criticism myself, her understanding of SdT totally misses the point. Every single time I have seen people complain about being told instead of being shown by a poor writer it was something like this (if necessary search that page for "show-don't" to find what I mean) or this.

So it is not about world building info dumps at all. It is entirely about being too poor a writer to communicate the abilities and emotions of one's characters. It is about merely stating that your protagonist is a good debater instead of introducing her by winning an argument. It is about thinking that your reader is too stupid to understand that the protagonist is sad when you simply write "Frodo cried" and instead writing something to the effect of "Frodo cried because he was sad, and he was sad because as you may not remember Gandalf had just fallen to his death, see previous page". It is quite simply about poor and lazy writing, in a way that is independent of cultural context except to the degree that some other cultures may not even have a tradition of fiction writing (e.g. if it is a culture without a written language).

But apparently everything has to be about Western privilege all the time; there is nothing in the universe that is not about Western privilege.
It's the same hubris that led the white Western establishment to assume its medicine, science, and values superior to all other cultures. We'll come back to that shortly.
Eh, no. A medicine is superior to other medicines if it heals more reliably, and a scientific methodology is superior to other scientific methodologies if it produces more reproducible and accurate descriptions of reality. There are things that demonstrably work (often including substances found in traditional healing herbs) and there are things that demonstrably don't (including the Western tradition of bloodletting). That is all there is to it, no Western or Eastern or whatever needed.

Also, apparently a story about a protagonist having an impact on the outside world is quite simply "colonialism". What? No, people interacting with each other, helping each other against a dark lord's attempt at world conquest, learning from each other isn't colonialism. Invading with an army and taking over other people's countries to exploit them, that is colonialism. Words have meanings. Or at least they should.

Sunday, October 15, 2017

Monga National Park

Monga National Park is c. one and a half hours east of Canberra along Kings Highway. It features wet sklerophyll / rainforest type habitats with many cryptogams.


We were there today in the hope of seeing Telopea mongaensis (Proteaceae) in flower. As can be seen in the above picture we were still a bit too early in the season, they are only just in bud. So far I have seen the Tasmanian species T. truncata, the New South Wales State Flower T. speciosissima, and, during a holiday in Victoria and southern New South Wales, T. oreades. The latter appears to be very similar and, I presume, most closely related to T. mongaensis.


What was in flower a lot in the same locality (the Waratah Walk from Mongarlowe River Picknick area) was Tasmannia lanceolata (Winteraceae), member of a 'basal' angiosperm clade, but of course it is far less spectacular.


This is the habitat; Telopea mongaensis is found particularly along the river.


The other attraction just a few hundred meters away is Penance Grove, which we had seen before. It is particularly known for its many tree ferns.


I am always fascinated by Dawsonia superba (Polytrichaceae), the largest moss in the world, which I believe is most easily accessed from Canberra by coming to Monga NP. I have written about it at least twice before, but I think this was the first time I saw it with young sporangia.

Tuesday, October 10, 2017

The world is so confusing sometimes

When will we finally reach peak gibberish in science spam?
Dear Author,
Formatting as in original - an auspicious start.
Journal of Proteomics & Bioinformatics greets you a good day!!!!
That's a new one, but at least it isn't "greetings of the day". Also, by the way, I really don't understand why my spam filter cannot finally figure out that anything that has more than one exclamation mark in a row can be binned immediately.
We are in shortfall of articles for successful release of Volume 10, Issue 10.
See this circle? That is my circle of caring. The fact that this alleged journal cannot fill its issue is about 5,000 kilometers outside of my circle of caring. So...
Is it possible for you to support us with your transcript for this issue before 30th October?
What do they mean with "transcript", which is usually a sheet showing university marks (grades)? Do they not even know the word manuscript?
If this is a short notice please do send 3-4 pages Short Commentary or Mini Review, and hope that a 5 pages article will not take much time for an eminent like you.
What does this sentence even mean? Help?
Also it will be very kind of you if you can acknowledge the receipt of this email and give your opinion to our proposal.
Better not, because if I honestly gave my opinion of their proposal there would have to be some bad language involved.
Best wishes,
Susan Williams
As usual, if this was written by somebody actually called Susan Williams... oh, excuse me, Susan Williams, then I will not only eat my hat but a whole stack of hats.

---

In completely unrelated news, why does GBIF suddenly use hexagons?


This looks as if somebody tries to draw in more people who enjoy strategy computer games, but it seems a bit odd given that spatial studies generally use square grid cells, either equal area or degree-based.

Saturday, October 7, 2017

No, science is not fundamentally broken just because one person had problems with their supervisor and prefers preprint servers

While on holidays I learned about an interesting case in food science, where it is suspected (at least according to some statisticians who have dived into the literature) that a very prominent researcher may have P-hacked, self-plagiarised, potentially reused the same dataset for several publications while making it appear as if they represented independent studies, and used self-citations to support statements that they don't. I find stories like those at the same time interesting, inspiring and very, very frustrating.

Interesting because, well, human nature and all that, and I am continually puzzled whether the people in question really think it won't come out in the end. Inspiring because this is how science self-corrects; not necessarily by individual scientists changing their minds (although that would be the ideal), but by open debate between scientists and careful re-examination of data. And frustrating because of exaggeration, over-generalisation, and naiveté about the solutions or alternatives to the problems that are identified. This post will mostly be about over-generalisation as found in the writings of one Jordan Anaya on Medium.

He evaluates and criticises the relevant food scientist's working practices, as do many others. But in doing so, he writes the following:
But I am interested in how academia selects for bad science, is free from any outside regulations that might prevent a crisis like the housing bubble, and how its power structure allows senior members to behave like dictators.
...
Science in academia is not about performing science, it is about your brand.
...
We are in the midst of a reproduciblity crisis in science,
...
all the problems science is currently facing
...
As it stands now, the wrong papers get published, the wrong researchers get funded. There is no incentive to share data or perform careful science. The only thing that matters is your brand, and your ability to leverage that brand into publications and grants, which circle back to feed the brand. If that means performing sloppy research, exaggerating results, and then refusing to acknowledge any errors, so be it.
...
Most of the literature is wrong, this is just a reminder that we need to be vigilant. It is also your daily reminder that peer review is useless and everyone should instead be preprinting their work.
I find this nothing short of astounding. Here is a bioinformatician generalising from his own bad experience in what appears to be one research group and one case in food science across all of academia and across all of scientific research. He does not write, "food science seems to have a problem", no, it is all of science, based on n=2.

When he writes that academia selects for bad science, can he really say with confidence that that is the case for, say, Australian entomologists? How would he know?

When he writes that it has no outside regulations, does he really mean to claim that it is not ultimately voters who decide through elected governments what kind of research will get funded? Is there so much money in cancer research in spite of or because of the wishes of the public? And this is before mentioning industry collaborations and industry-funded grants. (I will grant him that science isn't a democracy. The question is whether it could work as one, but that is beyond the scope of this post.)

When he writes that science as currently practiced in universities is not about science but about branding, can he know that this is the case for inorganic chemistry in southern Germany?

Do astronomy and phylogenetics really face a reproducibility crisis? I at least am not aware of that, and indeed I would be confident that if somebody were to repeat pretty much any phylogenetic study published in a serious journal with different molecular markers they would be able to reproduce all major results.

Is surface physics really facing "all the problems" that food science does? Again, how could Anaya even pretend to know?

How does he know that the wrong papers get published and the wrong researchers get funded in, say, plant ecology or archaeology? He claims that there is no incentive to share data - has he never heard of GBIF, Genbank, TreeBase or Dryad, or of all the journals that will not even accept your paper if you haven't deposited your data in a publicly available repository?

"Most of the literature is wrong." Seriously? Meaning I could pick a random article from a well-respected taxonomic, systematics or evolutionary biology journal and there would be a more than 50% chance that it is "wrong"? (And what qualifies as "wrong"? Does it mean this group of plants dispersed to Australia 16 million years ago instead of the currently accepted 15, or does it mean evolution is a lie meant to destroy Christianity?) And the alternative is to do away with all quality control whatsoever? But I am getting ahead of myself, this post is meant to be about over-generalisation, not solutions.

There is no doubt that the way science is being practiced leaves room for improvement, in particular in the areas of research funding, incentive structures, publishing, and recruitment. Okay, the same is probably true for any collaborative enterprise that humans have ever undertaken or will ever undertake, but tu quoque arguments don't change the fact that a lot can be criticised. I would have quite a few ideas for improvement myself.

Still, I hope one thing is clear: the fact that there are problems and some people acting in bad faith does not mean that an entire enterprise is broken. When we find that there are incentives for teachers to inflate grades we do not conclude that all of education is broken; when we find a certain percentage of police officers are racists we do not conclude that all criminals should go unpunished. And you may have heard that when we pour out the bathwater we usually take care not to pour the baby out with it.

Of course it is a question to be debated whether the entire system is broken. It could be. But from what I can see in plant systematics, ecology and related fields, this is so much hyperbole. In fact I am struggling to find a description of the act of dismissing all of science and the careful work of thousands of researchers with a mere "most of the literature is wrong" that is reasonably polite and does not involve phrases like "breath-taking arrogance".

It is also, of course, ridiculously irresponsible. Think of anti-vaccinationists, creationists, climate change denialists, alternative history cranks, expanding earthers, and any other set of conspiracy theorists. They already claim that science is all broken and the literature can't be trusted. Now they can say that a scientist frustrated with the review process in his field confirms it. Believe what you want, because the scientists can't be trusted!

Ye gods. Did the plain truth - science publishing and funding have serious issues that we need to tackle, and there are a few frauds just like in every other profession, but all in all we can trust most of our colleagues to be well-meaning and most of the literature to be useful - not sound sensationalist enough?

Sunday, October 1, 2017

Spring holidays 2017, final part 5

And we are back. Today we returned to Canberra passing through, among other things, the Putty Road between Wollemi and Yengo National Parks. It is a very nice, forested landscape, although it would be even better if there were one or two official lookouts in the southern part.


The above gives an impression of the road as seen from a little resting area along the way, in this case still very much towards the northern end.


About 40% or so of the way towards the south there is a locality called half-way house that was apparently once a cafe and petrol station. Now there is only a stall that sells coffee, soft drinks and snacks as well as metal and wood sculptures, thus the large statue above. I got a coffee.


Slightly before that spot we saw Conospermum taxifolium (Proteaceae) along the way. Their flowers are a bit different from the 'usual' Proteaceae that people are familiar with, such as Banksia or Grevillea. In Western Australia this genus is a striking part of the landscape in the form of the smoke-bushes, but the species here in New South Wales are less conspicuous.


Because this post does not have enough plants yet I am returning to one we saw at the beginning of the trip. It is among numerous photos that I did not upload because I was uncertain about the identification, but now I am reasonably optimistic that this is Darwinia procera (Myrtaceae), a rare and very localised species.

Thursday, September 28, 2017

Spring holidays 2017, part 4

Yesterday and today we continued to explore Myall Lakes National Park and its surroundings.


The village of Hawks Nest is very touristic, and so it is perhaps not a surprise that it has a spring flower walk that is recommended to tourists. And indeed there are masses of flannel flowers, but also other interesting plants.


At the other end of the National Park we today visited the lighthouse at Sugarloaf Point. Shown here is the view towards Seal Rocks, which unfortunately wrecked many ships even after the lighthouse was built, apparently due to prevailing wind conditions during one part of the year.


Botanically today's topic is climbers. Our first one is Kennedia rubicunda (Fabaceae), with surprisingly large red flowers.


I am reasonably certain that this would have to be Geitonoplesium cymosum (Smilaceae), a climbing monocot. The field guide calls it 'scrambling lily'.


Finally, the native passionflower Passiflora herbertiana (Passifloraceae) had been teasing us for a few days now, always there but never in flower. Today we finally found it in bloom, and that made my day!

Oops. Upon examination of the Flora of NSW key to Passiflora it turns out that this is introduced Passiflora subpeltata from Brazil, as it has large, leafy stipules. Also the Flora says that native P. herbertiana is red, which is really interesting because Fairley & Moore's Native Plants of the Sydney Region, which serves as my quick reference in the field during this trip, shows it as white.

Either way the one we saw is not native. Still, passionflowers are just something else, and they remind me of past field work in the Andes.