Re: [tlug] Journals, Authors and 'Free Peer Review'

["Stephen J. Turnbull" <turnbull.stephen.fw@example.com>]

Raymond Wan writes:

 > (Looking back at my previous message...)  We were 
 > hypothetically talking about me being a referee and looking 
 > at some paper which has not cited an ArXiv paper.  Do I 
 > point it out to the author?

That depends on the ArXiv paper.

I think two things are being confused here: the *scope* of the search
request, and the *contents* of the search response.  If you do a
search, and an ArXiv paper comes up within the scope of that search,
I'm say you should check that paper to see if it should be cited.

There *is* a cost to searches with wide scope: the effort needed to
filter crap or inappropriate material (eg, secondary sources like
newspaper articles and blog posts) from the results.  ArXiv papers
purport to be original research, I suppose, so you can't exclude them
on the basis that they're secondary -- you'd have to read them to
determine that.  However, if in your experience, or that of reliable
colleagues, ArXiv papers are 99.44% uncitable because they are crap or
because they are mere abstracts that provide no help in evaluating or
reproducing the results, narrow the search scope to exclude ArXiv.

If the engine doesn't allow that, of course it's perfectly OK to
ignore ArXiv items in the response.  The important thing is that
you've made a decision in principle and *before* searching to ignore
ArXiv with prejudice.  And yes, this is prejudice, but it's
unavoidable.  It's important to do it consciously and to occasionally
test the principle, to avoid descending into bigotry.

 > I don't think it's a rule but one does have to have a 
 > filter.  I wish we had the time to read everything in our 
 > field, but we don't.

But that's the whole point of algorithms, keyword fields in papers,
and search engines, to refine searches based on some criteria.

 > But this thread started (I think) because of my reply to Benjamin's
 > suggesting that it is really great.  I don't agree with that and I
 > think it's worth saying why.

That's fine.  But you haven't done so so far.  Instead, you've asked
why you should change your mind, and there have been two things,
whether you should cite and whether you should publish, both of which
you seem to be very negative about, but not for reasons that I really
understood.

 > Personally, I think these are all legitimate reasons, even 
 > the ones that I didn't single out above.  In particular, #2 
 > is actually a good reason that I didn't think about -- being 
 > scooped.

Scoops happens legitimately, too.  I recall a case where I was at a
seminar where an open problem was discussed.  A few months later, two
very senior researchers independently solved it in essentially the
same way.  One immediately submitted it as part of a longer paper
treating some related (but less important) problems to the leading
journal, where it was very quickly accepted.  The second immediately
submitted *his* paper containing only the main result to a "letters"
journal where it appeared in print approximately a year earlier.  It
took five years for the paper in the higher-status journal to catch up
in citations (now it's no contest, of course).

I was personally involved in a similar situation where I and the other
team submitted to the same journal (my serial number was one less than
theirs, nyaah, nyaah, nyaah).  I almost got screwed, because I was a
grad student and got a low-quality reviewer who said "great! publish!"
while my later-to-be coauthors (who already had a reputation) got a
high-quality reviewer.  Their resubmitted paper had far more content
than my accepted paper, but the editor put his foot down and wrote
"I'm sure Mr. Turnbull would have addressed the issues raised by the
referee as well as you did", so we ended up co-authors. :-)

Life isn't fair.  Not to me (I didn't get a good review), not to my
coauthors (I get credit for their work, and I'm not sure I would have
figured out the method they used for myself -- now that I've seen it
it seems obvious).

 > Someone beating another to publication can make what would 
 > have been a tier 1 publication to a tier 2 or tier 3.

Life is like that.  This happens all the time; I told you another of
my editor stories earlier.  He preferred a long lynching to a short
statement of an interesting problem.  But you're looking only at the
downside.  There are two upsides: one, you may deter somebody with
better connections from working on the problem and beating you out
based on connections or simply working faster or choosing a journal
with a short review queue.  Two, if you succeed at deterring them,
they save the effort and can work on something else.  That benefit
doesn't accrue to you but is socially valuable.

 > And speaking of the publisher, I don't really think they are 
 > that evil.

I assure you, they are that evil.  Just look at the horrible law they
are pushing through in the EU. :-(  Not to mention their profits.

 > Yes, they are expensive.  But if it's an open 
 > access publication charge, then it has been included in the 
 > project's budget.  And it's actually a small portion 
 > compared to the cost of running the research project itself. 

Speak for yourself, empirical guy.  For many social scientists,
especially the more theory- or policy-oriented variety, submission
fees or conference registration is most of the non-labor cost of
publication.  And grad students generally don't have much funding --
my boss was rich in grants but I went to the main conferences every
year on student loans.  At Japanese national universities, most grad
students are lucky to get JPY10,000 for a domestic conference and JPY
30,000 (30,000!!) for an international conference.

 >   It's probably worth nitpicking when we're running out of 
 > money at the end...  As for accessing the journals, the 
 > library does pay for it; again, it's part of their budget. 

Not any more: it's a prohibitively high share of their budget.  All
the universities in Japan are cutting back on everything but
ScienceDirect and friends because of the high cost of access to
journals.

 > As for peer review, I actually don't see it as being a slave 
 > to the publisher.

This I agree with.  However, I think that promptness bonuses to
referees, and things like that would be very good for most fields.
The publishers have no incentive to do this, so mostly they don't.

 > Anyway, that's a long-winded answer to your "Why not?".  At 
 > least for me, the "Why not" out-weighs the "Why".

I can understand why you personally might not submit to ArXiv, but
that discussion got mixed up in with the question of whether anybody
should search or cite work in ArXiv.  Referees aren't magicians, and
their incentives are almost entirely noblesse oblige.  Unrefereed
sources suffer from worse incentives, but they're not 100% unreliable.

Steve