By What Authority Doest Thou These Things?

 People who "publish" stuff on websites are self publishing.
 No one has reviewed their work.  This isn't the way science
 is done. (John Anderson)

For some reason this reminds me of a story.  In 1937 Albert 
Einstein sent a manuscript on gravitational waves to _Physical
Review_, and it was returned to him with a lengthy referee report
asking for clarifications.  As Pais recounts the story, 

  "Einstein was enraged and wrote to the editor that he
   objected to his paper being shown to colleagues prior 
   to publication.  The editor courteously replied that 
   refereeing was a procedure generally applied to all 
   papers submitted to his journal, adding that he regretted 
   Einstein may not have been aware of this custom.  Einstein 
   sent the paper to the Journal of the Franklin Institute
   and, apart from one brief note of rebuttal, never 
   published in the Physical Review again."  

I'm also reminded of Einstein's wry admission that every year between
1907 and 1914 he retracted everything he had written - and published -
the previous year.

Science certainly demands review, but the necessity of PRIOR
review is less clear.  Historically the current practice of 
peer review PRIOR to publication is a fairly recent innovation,
and developed hand-in-hand with the modern university system.
Academic institutions require some means of measuring the value 
and productivity of professors and researchers, and it usually 
isn't feasible for administrators to make such evaluations 
themselves.  Refereed academic journals thus play an important
role in administering the overall university system, allowing
administrators to use the number of papers an individual has 
published in sanctioned journals as a quantitative measure of 
his productivity - which obviously wouldn't be a valid measure
if people could publish at will.  The point is that the practice
of "prior review" is largely for administrative rather than 
purely scientific purposes.

The main *scientific* argument in favor of prior review is probably
that it filters out erroneous or low-quality work, thereby allowing
the larger scientific community to focus more clearly and not be
swamped by repetitious junk.  Although this still requires someone 
(the referees and editors) to sort out the junk from the valuable
work, it is probably a more efficient division of labor than 
having each individual do the sorting for himself.  Unfortunately, 
the potential benefit of this "filter" is somewhat offset by the
administrative function of academic publication, which has the 
effect of compelling many people to publish work of marginal 
value, not for the purpose of advancing science, but simply 
out of necessity to sustain their academic careers.

The main *personal* benefit of prior review is that it may save
a person from publishing an embarrassing mistake.  In this sense 
referees and editors certainly provide a valuable service to 
individual authors, but it isn't obvious that this reduced risk
of individual embarrassment contributes greatly to the overall
advancement of science.  (I suppose people are embarassed when 
they discover mistakes on their web pages or Usenet posts, but 
I'm not sure it matters to science.)

Another benefit of formal peer review (although it need not be
prior review) is that it actually compels at least a few people
who have an interest in the subject to examine a paper closely
and provide comments.  This is sometimes the ONLY feedback the 
author ever gets on his paper, even if it is published.  On the
other side of the coin, it can benefit the referees themselves
to carefully go through papers and hone their critical skills.

With regard to "publishing" on the web, I think it can be a
perfectly valid way of presenting scientific (and other) ideas
for review and criticism, and I suspect it will prove to be at 
least as effective as academic journals for disseminating information
and providing a free marketplace for ideas.  It may even be possible
to adapt web publishing to serve the administrative requirements
of academic institutions (e.g., count "hits" from a set of
sanctioned domains?).

Of course, those of us accustomed to traditional journals will need 
to adjust our thinking a bit.  For example, if someone says "X is 
true" and cites a paper in a refereed journal that also says "X is 
true", then we may have some confidence in the accuracy of the claim
 - even if we can't really judge for ourselves the merits of the 
argument.  We simply assume that if X was not true, the referees 
would have caught it.  In contrast, if someone cites a web page that 
gives an explanation for why X is true, we have to treat this somewhat 
differently.  If we can follow the argument on the web page and become 
convinced that, indeed, X is true, then the web page citation has 
been useful.  However, if we can't follow the argument, it's not so 
clear whether we should accept the truth of X simply on the authority 
of a web page.  Basically, an ordinary web page has no "authority".  
It either persuades or fails to persuade on its own merits.

The main problem with this is that modern science is so broad and 
deep that no one can hope to have sophisticated independent judgement
in all areas, and there may be times when we could make use of a
result without really understanding why it is true.  Of course, this
isn't a problem that many scholars would like to emphasize, because
it's considered bad form (at best) to make crucial use of a result
that you haven't personally fathomed.  Ostensibly, academic scholars
have no need of "authoritative" sources, but we all know that in fact
most scholars do rely on authority for many aspects of their work.
This suggests that there will always be a need for "prior review" of
the material presented in SOME forum, such as electronic journals,
many of which already exist.

Oddly enough, the area in which we have greatest need for authoritative 
assurance is in the truthfulness of reported experimental results - 
noting that there have been some unfortunate cases of falsified data 
in the biological sciences, for example -  but this is precisely the 
area in which referees are the least able to give assurances.  In 
general a referee can usually only do what we ourselves could do, 
namely, apply critical faculties to see if a paper makes some kind 
of sense, or is at least plausible.

Anyway, the thought I'd like to convey is this: When someone
denigrates "web pages" in general as a means of presenting scientific
thought, it seems to me they are really objecting to the fact that
ordinary web pages have no *authority* because they are not subject 
to prior review.  On one hand this is true, but on the other hand 
it bespeaks a valuation of "authority" over illumination that is
fundamentally contrary to the spirit of science.