FAQ

  • Server Configuration: Search engine partitions (07-18-2013)

Need assistance for CONTENTdm?

Server Configuration: Search engine partitions

Last review: July 18, 2013

Description: As the size of the search index grows, it may be necessary to create additional search partitions.

Resolution


CONTENTdm supports search partitions in order to accommodate very large search indexes. When the size of the index is greater than one billion bytes, additional partitions are required.

There are two ways that partitions can be created. Both methods require the use of the server maintenance script described in the Appendix of the Administration Guide for your platform (see http://www.contentdm.org/USC/downloads/install.php ).

1. Run the indexALL process. If the search index is completely rebuilt using the indexALL option, partitions will be created as needed. The index populates one partition until it reaches a size greater than 1 GB. At that point, a new partition will be created. Additional partitions will be created if necessary. Each of these partitions requires its own port. There is also a separate search process (referred to as the "aggregator") that needs an open port. This means that a two-partition configuration requires three ports in total.

2. Manually use the addPartition process. If a user attempts to build an index in CONTENTdm Administration (on Items > Index) and the index will exceed the one-billion-byte trigger point, an error will occur:

Last index failed: New partition needed.
Run manageCdmServices.exe -addPartition

or:

Last index failed: New partition needed.
Run cdmServices.pl -addPartition

Indexing in CONTENTdm Administration will be disabled until the addPartition process is completed. Running the above at the command line will stop the search engine, create a new search partition and create a search aggregator (only the first time). This additional partition and the aggregator both require dedicated ports. After the partition has been created, you can build indexes in CONTENTdm Administration.

PLEASE NOTE. Do not run addPartition unless CONTENTdm Administration has prompted you with one of the "New partition needed" error messages above. Doing so can cause a problem with indexing (the usual symptom is record duplication). If you have this problem, you will need to run indexALL to correct it.