"The purpose of publication in the world of research to develop cumulative, additive knowledge." (1) The grammatical error found in the previous sentence exemplifies an ongoing battle occurring in the realm of scholarly publishing. Found in an email listserv designed to enable conversation between design academics, this quote was part of Don Norman's message testing out ideas for his keynote speech at a conference scheduled five months in the future. Those who read the listsev are able to access Norman's ideas much sooner than those who attend the conference, even though Norman's notes clearly need some proofreading. Illustrated here is a central contention of the rapidly changing paradigms in scholarly publishing: the sometimes-flawed immediacy of sharing ideas online to advance public knowledge, versus the slow process of information hoarding before peer reviewed publication to advance one's career (and publisher's profit margin). The internet is enabling unprecedented interaction with the intellectual property (IP) of scholars. These affordances of digital information-sharing are not formally trusted (particularly in matters of tenure), yet we can trace the undeniable process of the old methods of evaluation creeping into new opportunities for dissemination. Soon to be more credible, open access to research data and papers can exist online with new IP models; however, embracing new approaches for information sharing and evaluation is integral to ultimately replacing the unreliable system of traditional peer review. 

Traditional academic publishing relies on the mechanism of peer review. Also known as refereeing, the peer review system is in place to ensure quality of work before formally adding it to the wealth of scholarly knowledge for future researchers to draw on. It is generally seen as the submission of a research paper by an author to a publisher, who then vets the work with a team of experts in the same field the author is writing about. The team of reviewers are not paid for their time; instead they participate ostensibly for the progress of knowledge. (2)

The history of peer review is important to note. It was established after World War II because of both the growth in science as well as the increase in pressure on scholars to publish. Before the war, university professors were expected mainly to teach, and publishing may have been seen as taking time away from the university. However, it became clear over time that the reputation of a university did not depend on the teaching abilities of the professors, but on the professors' scholarly reputation. (3) This of course leads to the current situation of "publish or perish", where the future a faculty member's career (especially pre-tenure) and advancement in a field depends largely on the peer reviewed publications they publish papers in.

The Cold War also significantly contributed to the status of academic publishing today. Federal spending on university research began shifting towards favoring economic competitiveness. The 1980 Patent and Trademark Amendments (Public Law 96-517) "allowed universities, nonprofit institutions, and small businesses to retain the property rights to inventions deriving from federally funded research." (4) As a direct result of this, "there is greater privatization of both the selection (the review process) and the results (intellectual property rights) of research." (5)

It is here that we can start to see how scholarly publishing became such a hot topic of intellectual property debate. It is for commercial competition that intellectual property is in place, not for the advancement of general knowledge for the public. But in keeping our nation's best interests in mind, should we not argue for the advancement of economic growth resulting from academic research? In the past, collaborations among scientists at competing universities or research institutions have lead to significant argument over who owns patentable information. Faculty are less likely to share information when it can cost their employer. Commercial and academic tensions are not new, but today universities, government and industry are deeply intertwined. (6) Sorting out intellectual property issues is complicated, and a barrier to the free publication of academic research.

The history and reality of the traditional peer review publication system in academia present many problems for us today. The first issue is that of the scholarly authors themselves. The current situation places extreme pressure on scholars to publish, and publish frequently. This can result in poor, hasty work, and equally dismal undergraduate teaching abilities. And since the reviewers of the papers are not paid for their review time, they are not really meant to be fraud-detectors. To identify fraud, one might have to replicate the results of the research, which would be very costly and time-consuming. The reviewers might also use the reviewed research as means for personal or professional gain–essentially claiming the research as their own in another venue. The system of depositing a pre-print (drafts of a research paper before peer review) of the paper before submitting to review can be seen as a method to prevent this. (7)

A deeper underlying problem of peer reviewed publishing is the idea that it is not truly about being peers. The trouble with great, new ideas is that they go against established norms, and reviewers might not have the intellectual capacity of those who have written the papers. Though likely an issue only a small percentage of the time, when it actually does count "we have a scientific social system in which intellectual pygmies are standing in judgement of giants." (8) If views conflict with reviewers' currently held beliefs, it could be very difficult for the new authors to dislodge those beliefs.

Take for example, Sokal's hoax. Sokal "decided to try a modest (though admittedly uncontrolled) experiment:
Would a leading North American journal of cultural studies
-- whose editorial collective includes such luminaries
as Fredric Jameson and Andrew Ross --
publish an article liberally salted with nonsense if
(a) it sounded good and (b) it flattered the editors' ideological
preconceptions?

The answer, unfortunately, is yes." (9)  Though he drew much criticism from the event, Sokal's experiment demonstrates a serious flaw in the peer review system.

The publish or perish model in scholarship is inherently flawed, but the advent of the internet is causing a true shift from the old ways of publishing. Now, information can be accessed by anyone, anywhere. Researchers can publish their own findings online, where anyone can publish conflicting results or opinions. The ability to digitize research results and make meaningful use of them is opening up a realm of possibility for the next generation of researchers. The argument for public, open access to research data is very strong–research can then be verified and critically examined, duplication of research work can be avoided, the research process can be accelerated through data sharing, and new findings can be made by merging data from multiple sources. (10) There are even more possibilities when we think of potential applications of data in computer-aided visualization, or how this public data might help propel information distribution further across the web.

Traditional print subscription-model scholarly journals using a peer review process are slowly being replaced, and in their place is a demand for open access to research papers and the data behind them. "Open access (OA) literature is digital, online, free of charge, and free of most copyright and licensing restrictions." (11) The internet allows truly open access to be available cheaply for the first time, and also is enabling the creation of new sharing and evaluation systems that authors and universities might one day adopt. Scholars have long since been dissatisfied with the realities of academic publishing, and now the internet finally gives them the ability to circumvent the publishing industry altogether. There are three main methods for doing this while ensuring open access: open access journals, institutional research repositories, and self-archiving. The peer review system remains in place for many open access journals, and these are the most frequently used vehicles transitioning researchers into the 21st century.

For many, retaining peer review online is very important to validating the internet as a legitimate information source. Peer review has been in place for decades for complex reasons, not all of which are negative. Universities who desire to leave the peer review process in place can make an easier transition online when keeping a similar model as the offline publishing mechanisms. Now there exists open access journals, as part of the push for open access to research articles. Advocates of the open access movement place emphasis on open access to peer-reviewed research articles and their pre-prints, because it is "literature that authors give to the world without expectation of payment." (12)

Open access journals are simply online journals, or offline journals which have moved online, that allow anyone access with no price barrier and minimal copyright barrier. They do not have to be peer reviewed–papers submitted through an editor or editorial board also count. (13) However, they frequently are peer reviewed, as it is still seen as the primary method of evaluating scholarly research–what separates scholarly information from the rest of the information available on the web.

Although there are strong arguments for the complete reconfiguration of intellectual property law (14), open access has solid legal basis in current copyright law. Open access requires the consent of the copyright holder, and therefore does not infringe upon copyright. Copyright holders agreeing to open access are usually consenting to unrestricted reading, downloading, copying, sharing, storing, printing, searching, linking and crawling of the full-text of the work. It is important to note that there is no piracy in open access journals, and they are not Napster for science. (15)

Even though open access journals are designed to accommodate current copyright law, it remains inconsistent with both digital data and scholarly works. Jessica Litman writes "The role of copyright in the dissemination of scholarly research is in many ways curious, since neither authors nor the entities that compensate them for their authorship are motivated by the incentives supplied by the copyright system. Rather, copyright is a bribe to entice professional publishers and printers to reproduce and distribute scholarly works." (16) It is imperative that we reevaluate intellectual property law in academia as we move forward into open access publishing. Large publishing companies are the main entities poised to lose money from open access, so it is those companies we need to keep most careful watch of.

Open access journals online are presenting a true challenge to traditional scholarly paper publishers. To put it more simply, the internet is destroying traditional information dissemination. Now that information can be communicated without a physical object, we don't need to pay for physical paper copies of peer reviewed scholarly journals. The publishing industry is what is suffering during this change, and what is fighting the hardest to remain relevant. Just recently, Merck paid academic publisher Elsevier to publish a fake peer reviewed journal that contained favorable data about their products. (17) This sort of unsavory cooperation between a research publisher and company further exemplifies the dangerous relationship that exists between academia and industry today. This relationship can have serious impact on our society–from jeopardizing health care to genuinely stunting progress in science.

Overall, open access journals have been proven to
improve citation impact and decrease citation latency. (18) This means
that ideas that are in open access journals get more exposure to people
wanting to build upon the ideas. Using software and public archives to contain citation records, it is now possible to track just how many citations are made. A key project in the technical side of open access, the Open Archives Initiative is a project that "develops and promotes interoperability
standards that aim to facilitate the efficient dissemination of
content." (19) This is exciting because not only does it make citations easy to find and utilize, but since citations are in a standard format, they can be harvested as meta-data by any other person, software system or other entity. This open access to the citation material makes it much easier to track and measure frequency of citations, and it duplicates information–essentially making many redundant backups of the data.

Without intellectual property or copyright law backing them up, we are left with the question of how to pay for the hard work that still must go into the publishing of open access journals. Open access journals are compared to broadcast television and radio stations in the way that they support themselves. Journals can have subsidies from universities or other professional societies. Some charge a processing fee on accepted articles. Still others have "income from other publications, advertising, priced add-ons, or auxiliary services." (20)  For supporters and advocates of open access, it is important to address the concern that authors may have to pay for entrance into an open access journal.

Open access journals are considered to be the "gold" road to open access, as compared to the "green" road to open access of institutional repositories, or open self-archives. There is some debate over which path is better, and whether there is case for mixing and matching the two. Guédon writes that self-archiving should only be done with already peer reviewed articles (20), which rather defeats the purpose of self-archiving, especially in terms of pre-prints or dissertations. In a critique of Guédon's article, Harnad argues


Harnad makes the important point that open access is about access and impact, not about reforming peer review or journal publishing. Although there is dire need for reform in these areas, open access was not created to address this need. Open access is the first battle to be fought in terms of scholarly publishing, while issues of publishing and peer review should come next. This will be a rocky road, since scholarly intellectual property and copyright issues with publishers come in both at the open access stage and at the peer review stage. Authors who participate in self-archiving can begin to address all of these issues by placing their pre-prints (pre peer review articles) and dissertations in a self-archive, which can also be known as an institutional repository, when it is maintained by research funders or universities.

Institutional repositories are another venue for open access to research data and research papers. The main difference between open access journals and institutional repositories is that the repositories are usually not peer reviewed. An institutional repository can contain research journal articles and digital versions of other academic papers such as theses and dissertations, but they can also include any other documents or artifacts, including text, images, moving images, mpegs and data sets. (22) The ability to host other documents other than peer reviewed papers is integral to the usefulness of the repository. By having open access to these archives, scholars allow others to access unpublished work that might be time sensitive or helpful to another's work. Of course, this brings up again the issue of copyright in academia and the nature of the commercialization of research. Crow explains:


The statement above illustrates the central challenge scholarly publishing faces in the advent of the internet, and it is not only open access, peer reviewed journals that are presenting the challenge. Digital research repositories are absolutely integral to ensuring a truly beneficial move to all-digital. These repositories will store information that can later be evaluated, searched, indexed, aggregated and made useful and meaningful. The power is in the aggregate, not just in the final polished papers. It is not the university's or individual's duty to preserve the financial success of the publishing industry, and we must place utmost importance on scholarship and dissemination of information.

Now that there are open-source solutions for institutional repositories (such as DSpace), the costs of maintaining a repository are small. One needs only web hosting space and minimal support from a web developer in set up and maintenance. Many universities already employing web designers and developers should have no issue finding someone up to the task of installing an open source solution, though one should be careful to monitor backup and ensure future storage of the database. 

In my evaluation, institutional repositories are absolutely integral for a university to maintain prestige and organization. Open access journals still face major challenges in being accepted as mainstream and useful for the tenure process. But repositories are useful in housing all of the material we are generating daily, and if nothing else, the information can be sorted out later. What is important is to collect it in an open format that can be searched, sorted, cited and above all–used.

As of 2005, adoption of digital repositories was still a large hurdle. Even MIT, who developed DSpace in collaboration with Hewlett-Packard, struggles to fill their database with content. They have not only hired a User Support manager to solicit content, but when that didn't work, they turned to a marketing professional. (24) The process of convincing faculty to self-archive in an institutional repository is no small task, and likely the biggest issue in adoption. 

Self-archiving content outside of an institutional repository is when a researcher hosts their work on a personal website. While this is certainly better than not self-archiving at all, it is far better to additionally publish to an institutional repository because that institution generally has long term data storage and archiving in mind. The data in personal websites is not securely monitored for long term archive, which is why it is best for an institution to maintain the process of backup and ensuring that the data is permanently stored.

In practice, there are two main philosophical sides to the usage of institutional repositories: those who view the repository as competition to traditional publishing, and those who see it as a supplement to traditional publishing. As outlined earlier by Crow, repositories by their very nature present competition to traditional scholarly publishers. But there is significant reason to see the repository as supplemental, at least in the short term. Clifford Lynch "fears that viewing IRs [institutional repositories] as instruments for undermining the economics of the current publishing system discounts their importance and reduces their ability to promote a broader spectrum of scholarly communication. Institutional repositories may better serve to disseminate the so-called 'grey literature': documents such as pamphlets, bulletins, visual conference presentations, and other materials that are typically ignored by traditional publishers." (25)

Keeping the possibilities open for institutional repositories is critical at this point. They have potential use as both a replacement or supplement to traditional publishing. Through future computational abilities, we might find ways to assist peer review in terms of fact and fraud checking. And while our society works out the cultural issues of the digital revolution, we can still be collecting and maintaining our scholarly work to advance progress in scholarship. One solution to the problem researchers face in collecting research data to archive could be the integration of a lifestream into the researcher's personal research area in a repository. Using a constantly updated, public archive of the researcher's edited experiences both online and off can take a lot of the technical pressure off of collection of data. It can then later be organized and tagged appropriately. The integration of personal information and interests among core research topics might be significant in bridging disciplines across repositories. As it exists now, lifestreaming is a form of data collection and self-policing that can be extremely useful to scholars.

Open access journals, personal self-archiving, and institutional repositories are the contemporary challengers to the traditional scholarly publishing system. Traditional publishers are not going down without a fight, and intellectual property issues will take a long time to dismantle or reconfigure. Though some are quite convinced that open access to research is inevitable, we are still left with the problem of evaluating the open access material. Though some open access journals are peer reviewed, they are largely not seen as reputable in universities, particularly during the tenure seeking process. Aside from encouraging scholars to make their research open access, we need to now figure out how to evaluate the quality, not just citation impact, of open access research and also consider alternate models to traditional peer review, which is imperfect itself. Still deeper is the underlying issue of more generally making information available online credible by traditional standards.

One solution to what is plaguing traditional publishers is that they can downsize to become providers of peer review service. Peer review is ultimately medium-independent. (26) This being the case, there really should be no concern about the migration of research data and papers online. We need to simply reconfigure the peer review process to evaluate open access content. This can be done either through formal publishers or through new communities. Replacements for peer review are an entirely separate issue from open access publishing, but still very worth considering, even briefly.

One study, measuring the quality of editorial peer review, concluded "Until we have properly defined the objectives of peer-review, it will remain almost impossible to assess or improve its effectiveness. The research needed to understand the broader effects of peer review poses many methodologic problems and would require the cooperation of many parts of the scientific community." (27) This alone gives us reason to question peer review, but is also indication of how difficult an overhaul of the system may be.

Two areas are of great interest in the future of evaluation online are that of the newspaper industry and the gaming industry. To return to the concept of the lifestream, a recent blog post looked at the possibility of news editors finding a new niche in filtering newsworthy items out of lifestreams. (28) Potentially, this concept can be applied to academic research. As research becomes more up-to-the-minute online, it might begin having more immediate effects, making it much more newsworthy. In that case, perhaps news editors will have reason to sort, filter and vet research lifestream information. One other area to consider when looking for replacements to the peer review model would be the gaming industry and game theory in general. We can inspect how winners are determined in various games, and perhaps apply these principles to evaluation of research data and papers. This vague suggestion requires much deeper investigation.

 To conclude, there are several separate issues in traditional scholarly publishing today, and as I have began to sort them out, I can identify multiple areas to address. First, printed, subscription based scholarly journals are quickly becoming replaced by open access to data and papers via open access journals, institutional repositories, or self-archiving personal sites. It is the very internet itself that is causing this shift of power away from the large scholarly publishing companies. There is little doubt that open access will replace content with price or copyright barriers. However, this does not mean that the very system of peer review is threatened at all, since it is medium independent. Peer review will continue to remain the benchmark of quality in academic publishing until a more useful mechanism of evaluation is identified. Peer review is unquestionably flawed, and now is potentially a good time to challenge the system due to the meltdown of the publishing industry.

Then again, if we, as scholars, are to ease the transition into digital, it can be in our best interest to bring credible peer review online first, before trying to find entirely new systems of evaluation. Making available pre-print research papers and raw data solves the issue of availability and immediacy of access to information, and nearly everyone can admit there is reason for some sort of evaluation system outside of self-policing. Peer review is the most adequate method at our convenient disposal. As computing power becomes greater and more people participate in sharing and filtering online, more evaluation methods will become clear. Commerce rating systems, commentary, news editors, game theory, and algorithmic systems might all yield fruit. But first, it is absolutely imperative to progress in academia to make all scholarly research and data accessible to everyone at no cost to the reader. Credible evaluation of this work is inevitable, as is the nature of the marketplace of ideas.

This conclusion provides ample opportunities for further investigation: a first step would be identifying exactly which online publications would be acceptable for
tenure and outlining exactly where progress is being made, if at all, in
universities considering open access journal articles. Next, the need for protection of research material before writing a proposal for grant funding needs to be examined. On the open access front, in depth investigation of the integration of lifestreams into institutional repositories is necessary. Lastly, we need alternative solutions to peer review, which may be found in data visualization/algorithmic solutions, newspapers, or game theory.