Batch Processing frequently asked questions

New projects

• Why can it take 90 days to evaluate/process a new project?
  
  Projects are placed in a queue in the order they arrive at OCLC. Within approximately two weeks of your project?s arrival, OCLC will provide you and your regional service provider with the name of the OCLC database specialist assigned to your project. The database specialist carefully defines the project to fit your needs and the particular format of your data. Since defining a project involves the creation and evaluation of sample files and can uncover unforeseen problems, the initial evaluation phase of a project may require up to 90 days. If there is a funding deadline or other time restriction for the project, please inform your OCLC regional service provider  and batchservices@oclc.org at the time you initiate the project.

Formats accepted

• How should my physical media be labeled?
  
  Your physical media (i.e., cartridge, tape, CD) should contain your OCLC institution symbol, your institution name and how many (Logical) records we expect to find during processing.  If you have more than one project with OCLC Batch Services, including your Project ID as well will help quickly identify how your records should be processed.
• What kind of 4mm or 8mm cartridges do you accept?
  

  Data Considerations:

  • Only place one file (MARC records) on the tape (no volume or header files).
  • Do not compress or write the records in an archiving format.
  • Use the extended ASCII character set.

  Media Considerations:

  • 4mm or 8mm DATs should be created using a DDS-1 or DDS-2 type drive.

   

Submitting files

• On the Batchload Order form, what is retain local data?
  
  When you batchload, your records are matched against the master records in WorldCat.  If matched, your symbol is attached to the record, and a copy of that master record is archived for you at OCLC.  If you were to ever need to extract these archived records, they would not contain any local data you may have added to the records you submitted unless you specify that you want to retain local data at the time of order.  You should check with your local system vendor to see what tags and types of local data might need to be retained.  There is no fee for this option.
• Is a sample for evaluation needed before sending live data?
  

  No, you may send live data in the first file.  Most local systems can output MARC data in a usable format so sending a sample is unnecessary unless you are uncertain of the usability of that data.

  If you have non-MARC data and would like to determine if OCLC can build MARC records for processing, a sample file can be submitted to batchservices@oclc.org.  Please indicate that it is a sample to simply determine if your data can be used for batch processing.  No action will be taken on this file, other than a review.

• Can I submit non-MARC data for batchload?
  

  Yes, OCLC has developed tools to build MARC records for batch processing, from patterned non-MARC data.  If your institution has bibliographic data in an patterned format (for example an Excel spreadsheet), an OCLC database specialist can work with you to build records suitable for processing.

• Can my vendor send files to OCLC to update my holdings?
  
  Yes, your cataloging vendor can send files directly to OCLC for batch processing.  You will need to communicate this to your vendor and you many need to supply them with instructions for submitting your data. 
• Explain the Submittal Represents options on the order form
  
  1.  Live Data means the file contains records that have already been evaluated and a setup has already been created, therefore the records are ready to be submitted for processing.?
  2. Sample ONLY means the records are only a sample and are not to be processed.  A Batchload Status Report will be written upon completion of the evaluation.
  3. Live Data for evaluation and subsequent processing means this is the first time a file has been sent, therefore must be evaluated, setup, and then submitted for processing.  A Batchload Project Definition will be written upon completion of the evaluation.
• Can we send files via a web browser like Internet Explorer?
  
  No.  This capability is not yet established.

Using file transfer protocol (FTP)

• How do I enable FTP?
  

  To electronically transfer files, contact your OCLC regional service provider and complete the following steps:

  • Complete a Batchload Order form.
  • For a one-time load, your file can be send through FTP by contacting OCLC Customer Services Division (CSD) for instructions on a one-time transfer via EDX. OCLC CSD can be reached at 1-800-848-5800 (choose option 1 from the menu) or by e-mail support@oclc.org.
  • An EDX Assessment form is needed if your library plans to submit files via FTP on an ongoing basis and does not already have an OCLC EDX account. 
  • The EDX User Guide provides more information.
• How many records can I send in one FTP file?
  
  The maximum file size that can be submitted for processing is 90,000 records.  Send files with no more than 90,000 records in each file.
• How many files can I send in one day using FTP?
  
  OCLC can accept up to 19 DATA files and 19 LABEL files per institution, per day via FTP.

Processing files

• How long does it take to process ongoing files?
  
  Once a project is setup, ongoing files will be processed within seven business days of receipt.
• Will I be contacted after my files are processed?
  
  No, currently there is no automated notification process to inform you your ongoing files have completed processing through batch.  Every file processed through batchload generates a unique Batch Number and a corresponding Batchload Processing Summary and Exception Report.  These reports are available on the OCLC Product Services Web (PSWeb) under Records and Reports.  You must have an OCLC authorization and password (Cataloging, FirstSearch, etc.) to pick up reports.  You many also contact your OCLC regional service provider to obtain these reports.

  You will be contacted if you have requested OCLC-MARC record output from your batchload project.  You will be given the file names and instruction on how to pick them up.

• Can I add original records to WorldCat through batchload?
  

  Yes. After records are processed through batchload algorithms to set institutions' holdings, non-matching bibliographic records can be further analyzed by OCLC staff to determine whether they are true originals to WorldCat and meet minimum cataloging requirements.

  To add original cataloging via batch, files of non-matching bibliographic records should:

  • Contain less than 2% machine-detectable MARC errors
  • Adhere to national standards for cataloging, including those for descriptive cataloging and forms of headings
  • Meet OCLC K-level standards for completeness, as described in OCLC Bibliographic Formats and Standards 
  • Contain fewer than 20% records that duplicate records in WorldCat

  OCLC database specialists evaluate records for adding to WorldCat on the basis of the entire file, not record-by-record.  Special collections (for example, manuscripts, theses, unique language sets) can be evaluated separately from an institution's general collection.

  Records that cannot be added to WorldCat can be returned to the institution for correction.  These corrected records can be added to WorldCat online by the institution, or returned to OCLC for further batch processing.

Unresolved records

• What happens to the records that are not matched?
  

  How your unresolved (non-matching) records are handled will vary depending on the type of data and/or type of project.  These options should be discussed with your OCLC regional service provider and the batchload database specialist assigned to your project.  OCLC may choose to treat unresolved or non-matching MARC bibliographic records from batch processing in any of these ways:

  • Do no further processing.
  • Provide you with a file of the records so that you can edit them further through your local system or OCLC cataloging interface (e.g., CatME). You can pickup your files using OCLC Product Services Web (PSWeb).  After editing, you may resubmit the records for batch processing or process them locally.
  • Return a text version of the unmatched records as an e-mail attachment (one-time or annual batch projects only).
  • Further analyze for suitability of adding as original cataloging to WorldCat.
• How do I know what the final unresolved records are?
  
  A Batchload Processing Summary and Exception log will be available for you via OCLC Product Services Web (PSWeb) for the initial pass through Batchload matching algorithms.  The unresolved from this batch are reprocessed through Batchload Duplicate Detection and Resolution (Batchload DDR).  Batchload DDR greatly increases the matching potential for Books format records.  A second Report and Exception log are posted to OCLC Product Services Web (PSWeb).  When you open the report, the very first line will read ?DUPLICATE DETECTION AND RESOLUTION?.  A file of your final non-matching records is also available for pickup on OCLC Product Services Web (PSWeb).
• What does ERR mean on my Batchloading Exceptions Report?
  
  The ERR notation indicates a record contains a machine detected MARC error.  If OCLC were to add any records marked ERR, they would be added with Encoding Level "E" and would require manual correction.  It should be noted that the machine detected MARC error(s) may not be the reason any particular record was not matched.  Some errors affect the matching process, other do not.  Please contact the OCLC database specialist assigned to your project to find out what types of errors are present in your unresolved records and what can be done to correct them.  Be sure to concentrate on the Exception Log related to the final unresolved.