LWW Assessment

Introduction

Below we consider the following issues:

• The database engines and formats currently in use by the LWW, and those anticipated in the near future.
• The components of the LWW "meta-base", including their content, structure, and engineering.
• Current and anticipated linkages and crossover amongst these components, including content, physical records, and document references.
• Engineering issues of read and write access for the ARP.

Database Formats

Advance

Within Advance documents are identified with DocIDs (below).

The Advance architecture is as follows. HTML form information is passed through UNIX pipes to the WebZ gateway, which then parses it and constructs a Z39.50 query. This is then sent in Z39.50 format to Advance, and in particular to the Z39.50 server which is a portion of Advance. Other portions of Advance then parse and execute the search, return DocIDs, format MARC records, retrieve URLs, etc. This information is then passed back to the Z39.50 server, sent in Z39.50 format back to WebZ, which then formats an HTML response.

Each different Advance corpus has its own set of indexed fields. Some (e.g. title) are common to all corpi, others (e.g. taxonomic classification in Biosis) are specialized to particular corpi. Some (e.g. subject) differ semantically among corpi, but can be identified with each other. To represent this, 22 common index "slots" available, which each special index terms mapped into a particular slot. The following table shows this mapping.
 

Single-word queries first check for pre-made keylists, otherwise the keylist is constructed at query time. Multiple-word queries are broken into multiple single-word queries, whose results are then combined with standard UNIX text manipulation commands (e.g. sort, merge, uniq).

Topic/Verity

Topic is structured as follows.

• A server can address multiple collections.
• A collection consists of multiple documents. In principle, the same document can exist in multiple collections. In practice (i.e. in SciSearch) this is not the case, but if it were, the user would have to link the records together manually.
• A document consists of fields, structured fields, and zones.
• Zones are optimized for indexing.
• Structured fields are like composite records or C structures.
• A style file determines the common format of documents within collections.
• The entire document can also be addressed as a whole for free-text retrieval. This is actually the fastest lookup method available.

This HTML/Explorer/Verity architecture is not unique to the LWW, but rather is used lab-wide in a number of different Explorer flavors for a number of different support environments. In the LWW, Verity is currently specialized to the SciSearch database from ISI (below).

Verity is highly optimized for text storage and retrieval, and retrieval of non-indexed text fields is feasible from any collection. Support for numeric data is weak, for example there is no indexing. Similarly, there is no relational information. Further, no linking capability is available among Verity collections.

ScienceServer/MPS

Similar to Verity, MPS is a dedicated information retrieval product on a proprietary database format. It supports a variety of boolean and other search capabilities (see http://www.fsconsult.com/products/mps-server.html).  The LWW currently has a binary license for MPS, but will be receiving a source license shortly.

Oracle

There are currently plans to bring up an instance of Oracle on a PC from Sandia National Laboratory in order to provide a dedicated environment on which SciSearch interaction with the VxInsight visualization product can be demonstrated. This machine will generally not be available to the ARP.

Alternatively, it has been suggested that an Oracle instance might be made available on the Sun Ultra 3000 soon-to-be development machine (see Server Configurations below), or on a new Linux machine.

Components of the LWW "Meta-Base"

General Issues

• Content and Structure: What kind of information is kept, from what sources, and in what formats or "granularities"?
• Keywords and Indexing: How are searches faciliated by either distinct or "generatable" keywords?
• Controlled Vocabularies: Are any used?
• Engineering and Interface: What database holds the information, and how can it thereby be accessed?
• Sizing and Updates: How big are the corpi? How often and in what proportion are they updated?
• Ancillary Data: Is other ancillary information associated with the component available, even if not stored directory in the corpus (e.g. historical SciSearch queries).
• Availability: Which LANL customers or associates can access the data?

Online Catalog

OPAC contains records for listings for all materials accessible through the Research Library which have MARC records. This includes:

• Physical Holdings: Books, journals, proceedings, and grey literature.
• Electronic Holdings: Electronic journals, electronic databases.

As a database of MARC records, the OPAC benefits from keywords, and the controlled vocabulary maintained by the Library of Congress (LC) system. Thus most ISBN and ISSN entries have correspoding LC numbers.

Scientific Catalogs

Inspec

"INSPEC (Information Service in Physics, Electrotechnology and Control) is the leading English-language citation database for the world's literature on all aspects of physics, electronics, and computing. INSPEC scans papers from approximately 4,200 journals, 1000 conferences, and other publications. INSPEC corresponds to the print publications Physics Abstracts, Electrical and Electronics Abstracts, and Computer and Control Abstracts.

Subjects: Astronomy; Chemistry; Computer Science; Engineering; Earth Sciences; Mathematics; Nuclear Information; Physics; Science (General/Popular)."

Biosis

"Citations, with abstracts, from Biological abstracts and Biological abstracts/RRM (reports, reviews, meetings). BIOSIS indexes approximately 6500 journals and 2000 meetings/year, as well as books and other materials. Subject coverage includes biological and biomedical sciences, botany, biochemistry, biophysics, biotechnology, medicine, public health, radiation biology, ecology and the environment.

Subjects: Biology/Genetics; Environment"

Engineering Index (EngInd)

"Engineering Index provides worldwide coverage of approximately 4,500 journals, and numerous conference proceedings, reports, books and dissertations. Over half a million records are for papers in published proceedings. Subjects covered are civil, energy, environmental, geological, and biological engineering; electrical, electronics, and control engineering; chemical, mining, metals, and fuel engineering; mechanical, automotive, nuclear, and aerospace engineering; and computers, robotics, and industrial robots. Engineering Index at LANL corresponds to the print publication Engineering Index, with additional coverage of conference papers and proceedings.

Subjects: Engineering; Earth Sciences; Nuclear Information; Reports"

DOE Engineering (DOEng)

"A multi-disciplinary database containing worldwide references to basic and applied scientific and technical research literature. The primary focus is on energy and related topics. The database includes references to publications provided by the U.S. Department of Energy, its contractors, and other government agencies; also information from the International Energy Agency's Energy Technology Data Exchange (ETDE) and the International Atomic Energy Agency's International Nuclear Information System (INIS). Approximately half of the references are from sources outside the United States. Abstracts are included for records from 1976 to the present. Approximately half of the references are to journal literature and 25% to technical report literature.

Subjects: Biology/Genetics; Chemistry; Computer Science; Defense; Environment; Engineering; Mathematics; Nuclear Information; Patents; Physics; Reports"

SciSearch

The Social SciSearch database is coming online soon, and holds a similar collection of social scientific literature. Here we focus on regular SciSearch only.

Keywords come in two varieties:

• Author-supplied keywords are passed directly through.
• "Keywords plus" are generated by ISI using an unpublished algorithm which works basically as follows. First a list of the referring papers which are present in the corpus, and then the union of the title words from those papers, is constructed. Those words occurring multiple times are then listed as keywords plus. It is unknown if generated keywords plus which are also keywords are retained in either or both lists.

SciSearch Architecture and Loads

• Nroff formatted raw files are ftp'ed from ISI.
• A parsing process then produces formatted text files quite similar to the output from e.g. SciSearch alerts. Article boundaries within the text files are delimitted with a tag.
• The text files store information on the number of times the article has been cited, and whether there is a link to an electronic version of the paper (e.g. through the EJournals facility, below). This information is updated within the text files monthly.
• With each load, an index of internal CitationIDs (see below) is also generated.
• At query time, these text files are formatted into HTML.
• The text files are then parsed and used to update two Verity Collections.
• The main collection stores everything except citation information.
• The cited collection stores author, title, volume, and journal, in addition to citations.

Information on which articles cite a given article are stored in the Verity Collections only, and not in the text files.

Conversely, detailed reference information on the articles cited in a particular article are stored only in the text files, whereas in the Verity collection this information is represented only indirectly through a list of appropriate CitationIDs, which can then be queried iteratively.

Historical Queries

• IP sending query
• Time stamp
• Query text
• Number of results

Standing alerts are not logges distinctly, but information about them is available. In particular, every week standing alerts generate appropriate standard queries, which are then recorded in the log files. Queries generated from alerts can be distinguished from queries generated manually by the fact that they include ranges of internal Verity record identifiers used to limit queries to the weeks in question.

Electronic Journals

The LWW has entered into a close relationship with Elsevier (not actually a publisher, but rather a distributor of multiple publishers such as Pergamon), other publishing companies, and FSConsult, in order to provide desktop access to a wide variety of electronic journals.  These parties all have a mutual interest in working together to develop these capabilities. We will focus specifically on Elsevier here, which is the major effort, but Academic Press is coming behind.

On a weekly basis, CDs are received from the publisher. A TOC file (in a tagged flat-file format) contains abstract information per issue on the CD, including authors, title, keywords, etc. Papers are available in PDF, SGML, and raw text. LWW code runs an indexing routine (both the TOC and the document text is indexed), creates SICIs, and updates the SICI database (see below).

No citation information is available in indexed form, although in principle it can be abstracted at least from the raw text copies of the papers, if not the tagged SGML versions.

A subject specialist LWW librarian makes LANL subject (category) classifications of all EJournals. This is how they are grouped on the web page.

EJournals are restricted to LANL and the Air Force Research Library, although discussions are ongoing with others (e.g. Stanford).

xxx.lanl.gov

Other Corpi

Unclassified Publications

"The Los Alamos Unclassified Publications Database is a multi-disciplinary database  containing references to basic and applied scientific and technical literature authored by the  staff of Los Alamos National Laboratory, its contractors, and collaborators."

LANL Technical Reports

Patents

Citation Analysis

• Ten most cited LANL article.
• Top ten journal in which LANL articles appear.
• Top ten categories in which LANL articles appear.
• Number of citations vs. number of LANL articles published.
• Some other statistics.

Summary

Current and Anticipated Linkages and Crossover

Content Crossover

Generally, however, we are informed by the following considerations.

• SciSearch is distinct from the others in that it is a very broad, but relatively shallow corpus, hitting the "top journals" in most fields of science.
• The Advance scientific corpi, on the other hand, are much more comprehensive within their own fields, including, for example, conference proceedings.
• Finally, the EJournals are also deep, but are effectively "samples" from the overall scientific literature, being grouped by publisher.

• Biosis-Inspec: Small overlap in medical physics, radiography, health physics, and simulation in common.
• Biosis-SciSearch: Moderate overlap of top biological literature.

Database Crossover

Importing xxx.lanl.gov

Movement to Topic

Document Reference Crossover

Document References

Document Object Identifier (DOI)

Serial Item Contribution Identifier (SICI)

SICIs using the "article wthin serial publication" format are contstructed at load-time in all of the the Advance, EJournal, and SciSearch databases. The syntax used in the Advance and EJournal databases is:

ISSN(yyyy[mm[dd]])vol[:issue[:part]]<page:title_code>control check_digit

• fixed-width font is literal.
• [x] is an optional item.I
• ISSN is the ISSSN.
• yyyy is the year.
• mm is the month.
• dd is the day.
• vol is the volume number, or otherwise the highest-level available "enumeration" information.
• issue is the second-level enumeration.
• part is the third-level enumeration.
• page is the first page number.
• title_code is a string consisting of the first letter of up to the first six words of the title.
• control is a constant.
• check_digit is a check digit.

• Title only
• Date only
• Enumeration only
• Neither date nor enum
• Neither date nor title

Due to the limitations of indexing information provided by ISI, SciSearch maintains its own format of the SICI, with syntax:

ISSN()vol[:issue[:part]]<page:>

DocIDs

Citation ID

lname initials # yyyy # journal_char vol # page

• lname is the last name of the first author.
• initials are the initials of the first author.
• yyyy is the publication year.
• journal_char is the first character of the journal title.
• vol is the volume.
• page is the starting page number.

URLs

When the EJournals are loaded, a "SICI database" is updated, holding <SICI,URL> pairs, with six entries for each URL.

URL information for journals which are available remotely to the LWW is also available. A table holds publisher-specific URL templates which can be instantiated in predictable ways using partial SICI information (e.g. ISSN and enumeration).

Both of these tables are made available to Advance, and then sent to SciSearch.

Document Reference Crossover

Engineering Issues

ARP Read/Write Access

Read Access

• Advance: Read access in Advance is possible, but would likely be quite difficult, involving either interfacing with the WebZ server or some other portion of Advance, or with the untested and unproven SQL interface available through Universe.
• SciSearch: Access to SciSearch information is available either through the Verity API calls to the Verity collections, or directly to the text files. Verity querying will be necessary to retrieve full citation information. It is questionable whether massive extraction from the Collections will affect performance inordinately
• MPS: Information about EJournals necessary for the ARP is best retrieved through the text TOC files associated with each loaded CD.

Write Access

• Advance: Again, interfacing directly to the PIC environment will be difficult, and the SQL capability is unknown.
• Verity: While we could establish distinct ARP Verity collections, little if any interaction among collections in a server are supported. Further, there is poor to no support for numeric data and no relational capability.
• ScienceServer: Little is known of this architecture, although its user interface is quite similar to Verity.

Server Configuration and Loading

Further details are available at http://lib-www.lanl.gov/~mlbm/libnet/index.html, currently in draft form.

Acknowledgements