All of us in this business know how bruising it can be to get referee reports on our research papers. Even when the referees are gentle and helpful it still is embarrassing to discover the errors we’ve made in our texts. On the other hand, it’s a good thing they were found before publication. Of course, in double-blind peer review we are relying on the system for gate-keeping to make certain publications are original and rigorous and also that the research is replicable. That’s really the role of the referee.
Unfortunately, often referees produce really unpleasant reports. I know many of us have experienced this because we talk about it all the time at conferences and in the hallways at school. I really don’t know why referees find it necessary to be so rude, but it happens for sure. When I can, as editor, I edit the referee reports making healthy use of ellipses to leave out vitriol and just give the authors the critical commentary needed to revise the paper.
All of this is prelude, of course, for the most outrageous peer review I’ve ever received. Of course, I do not know who the referee was, although I suspect when I get done here, whoever it was, will recognize the points I’m going to make. Oh well, that’s why this is a blog ….
Clearly, whoever it was either knew who I was to begin with or sussed it out from the methodology and context. Let’s face it, in a small field like knowledge organization we all know each other well enough to recognize each other’s writing. And often we have the situation of being one of only two or three authors in an area, which leads to a lot of self-citation in order to cite the most relevant prior research. Be that as it may, whoever it was reads my editorials in Knowledge Organization–the ones that are simple bibliometric analyses. I have limited space of course, and I try to focus the editorial as commentary on the evolution of the domain. But I also deliberately state that I am just giving a few simple metrics. I always post my data files here, and encourage others to download those files and take the research further. So far as I know, nobody has ever done that.
But, please–here is what this anonymous blind reviewer wrote: “Two or three observations and we move on … or maybe there is really nothing more that can be said? … Are we supposed to do the analysis ourselves?”
Umm, yes, that would be the idea!
Okay, I took the point and extended the analysis into excruciating detail in the final version of the manuscript. But, yes, the idea of science is that we all work on the same problem sets to try to advance understanding! Please!
But, the title of this post comes from the best, which I’ve saved for last. I had quoted a paper that I believe will be seen as seminal by Joe Tennis (you know, author on subject ontogeny, knowledge organization domain analysis, ethics, etc., and current president of ISKO?). Not wanting to copy wholesale (because, that’s not what we do in science, is it?) I made reference to the article and cited one or two points. Here is how this referee indicated more detail would be useful: “but most of us … will
not necessarily have time to read Tennis.”
Well, I want to reply, “you’d better find the time!” For goodness sakes, people, this is why we have text referencing. If you haven’t read the paper being referred to, you are supposed to go do that!
We have received some fascinating manuscripts for Knowledge Organization recently. Some of them come from outside KO but with a bent toward information, others come from disciplines completely apart from ours. All of them speak directly to our domain about the phenomena that comprise our science. All of them were problematic in peer-review.
If we are going to thrive as a science principally concerned with knowledge, then we must be open to learning what others have to teach us about our own phenomena. There is, I suppose, a fine line between accepting work that does not fit into our discussion because it fails to acknowledge our domain, on the one hand, and work that in essence contributes to our domain even if the authors have naively underrepresented their intersection with our domain. Was that diplomatic enough?
We have to show these authors where the intersections are, we have to point out the boundary objects. Then we have to suggest how the papers can be reconfigured to speak more cogently to our domain. It is, curiously, the realization of epistemology within our science. It is how we approach true interdisciplinarity.
And then we have to take it one step further by stopping ourselves from wandering off aimlessly or turning our backs on new ideas just because they did not spring from within a culture of classification. We have to reach across those boundaries and invite conversation. It is hard work, but it is essential for the advance of KO as a domain.
Knowledge organization the activity–that is to say, classification, indexing, metadata and systems for their use–has been around forever. Academic development of systems for taxonomy trace to Linnaeus in the 18th century, indexing traces often to Callimachus in the third century B.C., cataloging rules have various forbears from the early printers to the French Revolution to Panizzi, Jewett, and Cutter in the mid-19th century. The application of scientific method to the problems of knowledge organization, arguably dates from the second decade of the 20th century when the Graduate Library School at the University of Chicago was created specifically for that purpose. It is from this stream that the science of fat-cards, for example, yielded understanding of sampling from frames with unequal probabilities. And it was the application of that method to the problem of instantiation in the catalog that helped unravel the problem of disambiguation created by KOSs that did not comprehend the parameters of instantiation.
Knowledge organization the science, articulated by Dahlberg in the second half of the 20th century is fairly recent but seems to be thriving, according to all accounts, with growing international conferences and globalization. There has been some confusion over the terminology. Is information organization the same as knowledge organization? Some authors say they are the same, some say there are slight differences. It doesn’t help that a key monograph by Svenonius uses “information organization” as does a core textbook by Taylor and others. Here is one potential tipping point. We must insist on the use of the correct terminology. We receive manuscripts for publication in the journal Knowledge Organization, believe it or not, that use the term information organization. We change it in editing; always. We have to insist, however, in all of the academic areas in which knowledge organization is seen either as a subset or a neighboring discipline.
My research group changed its name this week to Knowledge Organization Research Group, or KOrg for short. I was amused at the opening day of school two weeks ago when, during a doctoral orientation luncheon (which usually involves the whole school), all of our doctoral students stood up and announced they were studying KO. This week I chuckled (or should I say “lol”) when I pulled up the ASIST program and say an entire panel labelled knowledge organization. Two small wins. Not yet a tipping point.
The other place where this sort of precision is critical is in our insistance that knowledge organization and knowledge management are not the same thing. They are not, and they must not be confused. ISKO conferences must be clear about why they accept papers on that other subject (I’m avoiding the keywords here). Knowledge Organization, despite our emails to indexers and my editorials, continues to be indexed as that other subject. We must intercede, if we want to reach the tipping point. In the meantime, we have been adding keywords to our articles to give the indexers hints. (We are not using author stipulated keywords, which amusingly rarely are precise or even applicable. Instead we run each text through a term frequency tool that shows us which keywords really are in the text.)
Svenonius, Elaine. 2000. The intellectual foundation of information organization. Cambridge, Mass.: MIT Press.
Taylor, Arlene G. and Daniel P. Joudrey. 2008. The organization of information. 3rd ed. Library and information science text series. Westport, Conn.: Libraries Unlimited.
I’ve been toying with WordStat™ software from Provalis Research again. It is very useful for the kind of qualitative analysis required in domain analysis. One valuable tool in the content analysis package is a KWIC index. Ancient students of KO will recognize that acronym for “Keyword-in-Context,” a kind of indexing once thought potentially fruitful. Here is an example including three “contexts” for the word “model” from ISKO 13’s proceedings.
||of information retrieval systems
|A reference ontology for biomedical informatics: the Foundational
|Towards a Comprehensive
||of the Cognitive Process and the Mechanisms of Individual Sensemaking
As you see, it is very useful for comprehending the precise context of those big words that show up in the center of word clouds or the foreground of MDS plots.
However, the interesting thing I’ve just learned is that most of the presence of the term “information science” in our domain comes not from the keywords in research papers, but rather from the title of the third most cited journal in our domain JASIST (forgive me for not spelling out here, and using that term again). Thus it is not that that term is a topic of critical interest, rather it is that as much as 20% of our research appears in a competing journal.
If our science is going to continue to thrive and grow, our authors need to stop sending their research to competing journals. Better a world in which our journal Knowledge Organization has to split into an A for ontology and a B for epistemology and a C for domain analysis, etc., than one in which the dispersion of our science hinders exploitative power and weakens the scientific structure of our domain.
I famously wring my metaphorical hands about the number of authors who submit manuscripts to Knowledge Organization reporting research that is topically relevant, but showing absolutely no inculcation in the theories or values of the science of KO. Emotions range from demoralized to furious on these occasions. Fortunately, rational academic policies dictate manuscript acceptance, and in almost all cases we return these errant papers to the authors with instructions to go do their homework. Some of them do, happily.
I am in the midst of a domain analysis of the 75 papers presented at the recent ISKO International Conference in Krakow (http://www.isko2014.confer.uj.edu.pl/en_GB/-start). The complete results of that analysis will appear in an editorial in a future issue of KO. But the interesting thing I am seeing this time is that there is, indeed, a core of knowledge organization. Seventy-five papers, 1200-some citations, from 20 countries, citing over 400 journal articles, 300 books and 200 anthologies. And yet, most of the citations are to a tightly-knit intellectually coherent core of KO. Most journal citations by far (44%) are to Knowledge Organization, the majority of conference papers cited are in ISKO international conferences or regional chapter conferences, and the most-cited monographs are by Hjørland and Ranganathan.
It is good news, that there is such a strong and resilient and theoretically useful core of knowledge organization. The challenge, it seems, is to require those interloping into our topical areas to encounter our theoretical base.
I recently completed a rich analysis of the entirety of American Documentation in order to trace the evolution of the concept of a concept across that era of the growth of the emerging field of information science. I wrote a short paper on the subject for CAIS 2014 (available here: http://www.cais-acsi.ca/ojs/index.php/cais/issue/current.
The “abstract” is this: A core entity of information science is the “concept.” Agreement on the basic definition as a mental construct representing a concrete instance, conceals divergence in understanding of the nuances. A case study of the domain’s nascent era represented by American Documentation reveals some of the contours of the terms evolution.
There were lots of fun things to be encountered in those years of AD, and I was going to upload some photos of things like the rapid selector and Termatrex and so on, until I went to do so and found all of those “further reproduction prohibited” notices. Oh well. The whole run is available to ASIST members in the ASIST Digital Library.
I thought it was fascinating to see how interwoven knowledge organization was in those early days of documentation into information science. There was a lengthy evolution of something called “the duality concept,” which was an expression of the dichotomies between known-fact and browsing, between simple and complex terminology, and thus between isolate and hierarchy.
Stay tuned: a lengthy journal article is forthcoming.