Re: [tlug] Journals, Authors and 'Free Peer Review'

[Raymond Wan <rwan.kyoto@example.com>]

Hi,

I've been trying the "let's agree to disagree" tact for a 
while but I guess I was doing too indirectly.  I'll try to 
be a bit brief but, indeed, "let's agree to disagree" should 
be the overarching summary...


On Monday, October 01, 2018 09:26 PM, Stephen J. Turnbull wrote:
> Raymond Wan writes:
>
>   > (Looking back at my previous message...)  We were
>   > hypothetically talking about me being a referee and looking
>   > at some paper which has not cited an ArXiv paper.  Do I
>   > point it out to the author?
...
> If the engine doesn't allow that, of course it's perfectly OK to
> ignore ArXiv items in the response.  The important thing is that
> you've made a decision in principle and *before* searching to ignore
> ArXiv with prejudice.  And yes, this is prejudice, but it's
> unavoidable.  It's important to do it consciously and to occasionally
> test the principle, to avoid descending into bigotry.


Hmmmmmm, I fear if I say "no", this whole discussion will 
then descend into a "but you said...".  So, to satisfy the 
trap that you set for me, I'll go with you and answer your 
accusation with a "yes".

That is, I am explicitly ignoring papers in ArXiv.

My reasons are as follows, but it's ok if you don't agree 
with them.  In general, I don't trust work that hasn't gone 
through peer review.  Peer review might be flawed but I 
would surmise that a peer reviewer might check a paper far 
better than someone citing it.  S/he has his/her reputation 
[with the journal] at stake.  And his/her entire field.  Is 
this being cowardly and literally "passing the buck"?  I've 
created another trap and sprang it for you as bonus...my 
answer is "yes".

Do I make exceptions to this "rule"?  Yes, of course.  An 
actual example from my former area of compression is the 
Burrows-Wheeler transform.  As far as I can remember, the 
paper to cite was a 1994 technical report from the company 
which the two authors worked at.  Yes, this wasn't peer 
reviewed.  But the decision to cite it was "easy" for me 
because by the time I came around to the research area, this 
paper was highly cited.

In fact, it is possible in my current area of research to 
have some method that many people use.  And people then cite 
the software (i.e., the GitHub repository).  When the paper 
finally appears, even if it was ArXiv, people will cite that.

So, if you're trying to ask me if my prejudice has 
exceptions, then yes as well.  At least 3 types of 
manuscripts appear in preprint servers (let's generalize as 
what I'm talking about is preprint servers and not 
necessarily ArXiv.  For example, in my area, we use bioRxiv 
but I didn't mention that).

1)  The authors chose to not carry it further for publication.
2)  The work was actually rejected by somewhere and the 
authors want to plant a metaphorical flag to indicate the 
date that they did it.  Sometimes it is obvious because the 
manuscript has been formatted according to a conference or 
journal's specifications.
3)  The work is going to submit it somewhere soon.

When you look at a paper in a preprint server, you don't 
know which of these 3 it is.  1 + 3 means that it hasn't 
gone through peer review.  In fact, who knows when 3 will 
actually happen.  2, on the other hand, means someone 
rejected it.

What would help me (i.e., a step in the right direction, but 
not necessarily would change my mind immediately) would be 
if the authors submitting it indicate which one of the 3, or 
if there is another reason, that they are submitting it.

We mentioned about how nicely the papers may be formatted 
it.  They might be, but they don't have to be. 
Nevertheless, this piece of information is not mentioned. 
That is, why did they submit it to a preprint server.


>   > But this thread started (I think) because of my reply to Benjamin's
>   > suggesting that it is really great.  I don't agree with that and I
>   > think it's worth saying why.
> That's fine.  But you haven't done so so far.  Instead, you've asked
> why you should change your mind, and there have been two things,
> whether you should cite and whether you should publish, both of which
> you seem to be very negative about, but not for reasons that I really
> understood.


Well, no, I never asked why I should change my mind.  If you 
and Benjamin left me alone, I would have been quite ok with it.

Again, I have no intention to change his or your minds. 
Surely, if the 3 of us were co-authors on a piece of work 
then, alas, we need to find a way to resolve it.  But lucky 
for the 3 of us :-D, we aren't working together so this is 
really just a friendly discussion.

For anyone who is still reading this, you two present your 
reasons, I present mine.  If someone looks at it and says, 
"Ray is full of fluff in his reasoning"...that's ok!

I will admit that I am talking about preprint servers as I 
currently view them.  Perhaps they will change some day to 
address the issues that I raise but also in that survey I 
mentioned (which, for what it's worth, I did answer before 
seeing what everyone else said -- so, yes, I was curious and 
wondered if my views were alone).

Many universities also have graduate students submitting 
technical reports into their local university's electronic 
repository.  These are usually also not peer-reviewed and, 
yes, I would consider them no different than a preprint 
server.  Usually, they will then get peer-reviewed in the 
form of thesis examination.  In fact, I think supervisors 
ask their students to do such technical reports to get them 
writing...  ArXiv is hosted by well-known university; 
bioRxiv is as well...I don't think that changes things.


>   > Personally, I think these are all legitimate reasons, even
>   > the ones that I didn't single out above.  In particular, #2
>   > is actually a good reason that I didn't think about -- being
>   > scooped.
> Scoops happens legitimately, too.  I recall a case where I was at a
> seminar where an open problem was discussed.  A few months later, two
> very senior researchers independently solved it in essentially the
>
> Life isn't fair.  Not to me (I didn't get a good review), 
not to my
>
> Life is like that.  This happens all the time; I told you another of
> my editor stories earlier.  He preferred a long lynching to a short


I cut what you said above because I totally agree.  And have 
been scooped or know of people who have been.

Legitimate scooping is...well...life.  I mean, with the 
number of people on this planet, the odds that two (at 
least!) are working on the same problem is very likely.  The 
irony is that the work has to be peer reviewed.  So, the 
editor needs to find people who know the work well enough to 
comment.  So, it's impossible for one group to be working in 
complete solitude.

Preprint servers aside, it is possible to give a poster 
session and talk about one's work.  To someone who knows 
your area, a single poster is enough for someone to scoop you.

We've digressed back to "why not submit" and not "why not 
cite", but that is why I wouldn't submit *and* why I 
wouldn't talk about my work too prematurely unless it's too 
someone I trust.  I've been to seminars by an external 
speaker who says that it is unpublished work.  And guess 
what...people (usually students :-) ) start taking out their 
smartphones and snapping pictures.  It reached a point where 
the now very angry speaker asked them to stop.

Anyway, if it's a serious project...that is, not something 
that we're doing "for fun", then we do need to be careful. 
Yes, you are absolutely correct.  "That's life!"  But we 
should also do our part and not be careless.


>   > Yes, they are expensive.  But if it's an open
>   > access publication charge, then it has been included in the
>   > project's budget.  And it's actually a small portion
>   > compared to the cost of running the research project itself.
> Speak for yourself, empirical guy.  For many social scientists,
> especially the more theory- or policy-oriented variety, submission


Well, publication charges are at least predictable.  In 
terms of how many and amount for each publication.


> year on student loans.  At Japanese national universities, most grad
> students are lucky to get JPY10,000 for a domestic conference and JPY
> 30,000 (30,000!!) for an international conference.


For what it's worth, in the life sciences, conference papers 
are not peer reviewed.  So students rarely get to go to them 
at all.


>   >   It's probably worth nitpicking when we're running out of
>   > money at the end...  As for accessing the journals, the
>   > library does pay for it; again, it's part of their budget.
> Not any more: it's a prohibitively high share of their budget.  All
> the universities in Japan are cutting back on everything but
> ScienceDirect and friends because of the high cost of access to
> journals.


Yes.  That is true everywhere.  I think each side (library 
vs publisher) is trying to figure out their bottom line. 
Once the publisher pushes too far and libraries start not 
subscribing, then watch them beg to have libraries want them 
again!  :-)

We can't submit to a journal we have no access to...  It 
would be funny to not be able to see our own accepted 
submission...but I guess that could happen.

I don't know how much of their budget goes to journals, but 
I do see less and less books being purchased...


> I can understand why you personally might not submit to ArXiv, but
> that discussion got mixed up in with the question of whether anybody
> should search or cite work in ArXiv.  Referees aren't magicians, and
> their incentives are almost entirely noblesse oblige.  Unrefereed
> sources suffer from worse incentives, but they're not 100% unreliable.


Yes.  I agree with this.  Submitting and citing are two 
separate issues and this thread inadvertently merged the 
two.  Though, actually, I didn't "question whether anybody 
should search or cite work in ArXiv (or a preprint server)". 
 That is each person's choice and I never have the 
attention to ask somewhat what they should or shouldn't do. 
I only said that I wouldn't.

But there are exceptions, as noted above.  Another obvious 
exception that I alluded to earlier is when there are 
co-authors involved since whatever is decided needed to be 
agreed upon by all those involved.  That is true of both 
submitting and citing.

Of course even peer reviewed work has its faults.  If it 
didn't, the word "retraction" wouldn't exist.

Ray