DRS 2 Web Admin Architecture

Jonathan Kennedy, HUL/OIS, 10/6/2010

1. Introduction......................................................................................................................2 2. Services Architecture.......................................................................................................2 3. New Services...................................................................................................................2 4. Technologies Used...........................................................................................................3 5. Ingest Control...................................................................................................................3 6. Solr Searching..................................................................................................................4 7. Editing Sets......................................................................................................................4 8. Future Work.....................................................................................................................5

1. Introduction
The DRS Web Admin will undergo a complete rewrite as part of the DRS 2 project. The next-generation Web Admin will support the new DRS 2 data model which introduces high-level objects and an expanded set of preservation and descriptive metadata for DRS content. Additionally, new features will be introduced, such as: improved searching, reporting, metadata editing, and user management. Please reference the DRS 2 Web Admin Architecture diagram for a visual representation of the architecture.

2. Services Architecture
The DRS 2.1 phase introduced a services-oriented architecture to the DRS. The DRS Services expose the functionality necessary to interact with DRS content. This functionality includes typical CRUD operations (Create, Read, Update, and Delete) as well as more specialized functionality. As described in the Architecture Diagram, the DRS Web Admin will not interact directly with the file system, Oracle database, solr index, or other underlying data sources. Instead, access to the underlying data sources will be mediated by the DRS Services layer which is responsible for ensuring that all DRS data sources are kept in a consistent state. It is currently thought that the functionality required by the Web Admin is sufficiently coarse in scope so that direct access to the underlying DRS data sources is not necessary for efficient operation. New service calls will need to be added to the DRS Services to enable the required Web Admin functionality.

3. New Services
The following new service calls have been identified for addition to the DRS Services to support Web Admin functionality. New services may be added as the need for additional functionality is identified.
Content-related Services View View View View Object File Batch Event

Update Object Metadata Update File Metadata Update Object Structure Delete Object Delete File Move File Add OCR

Administrative Services Add Content Model Add Format Add Owner Code Add Billing Code Add User Update Content Model Update Format Update Owner Code Update Billing Code Update User Start/Stop Ingest

The functions described here will be added to the existing DRS Services Web Services spec which describes the arguments and return values for each service call. The addition of these services will also require updates to the drs2_util package which exposes a Java API for calling the services.

4. Technologies Used
The Web Admin UI will be written using the Struts 2 API along with standard JSPs. A Javascipt toolkit will need to be selected to support AJAX-style functionality. Calls to the DRS Services will use the drs2_util CallService API. Solr will be used for searching and for generating the values used in report generation. Other toolkits that may lend additional reporting features will be evaluated.

5. Ingest Control
The Web Admin will support new features to permit control of the DRS Ingest process. A new service call will be added to support the remote starting and stopping of the Ingest Service. Ingest processing may be stopped for individual owner codes, billing codes, HUIDs, or drop boxes.

A new table will be added to the Oracle database will be required to maintain the state and restrictions of the Ingest Service.
INGEST_RESTRICTIONS PK RESTRICTION_ID TYPE VALUE START_DATE END_DATE

New response codes will be added to the DRS Ingest Service which will instruct inform the Batch Loader of the suspension in the event that the Batch Loader attempts to process a batch for a suspended entity. The Batch Loader will set an internal delay of 5 minutes before it attempts to load the offending batch again, and then it will move on to processing the next batch in the queue. Updates to the Batch Loader will be required to support this new functionality.

6. Solr Searching
The Web Admin will present the user with one or more interfaces to accept search terms and construct a query appropriate for Solr. Communication with the Solr search service will occur over HTTP. In order to ensure that users can search for only the content that belongs to them, direct access to the Solr server will be denied. Instead, all queries will be routed through a Solr Proxy which lives within the DRS Web Admin and will add the necessary parameters to the Solr query in order to restrict the results to the appropriate content.

7. Editing Sets
The Web Admin will support the editing of multiple objects and files in batch. Since updating many files at once could take a long time, these operations will be performed asynchronously. The user will submit the update via the Web Admin which will pass on the information about the batch operation to the DRS Services. The DRS Services will respond with the successful transmission of the request and begin processing asynchronously. The user will be presented with a message indicating that the process was successfully kicked off and that changes to the data may not immediately be visible. Should we lock files for further updating while they are being updated as part of a set?

Should we limit the number of files that can be updated in batch? A file could be locked for a long time if it is part of a very large set. Any updates to object or file metadata will require a change to the appropriate descriptor file. Since updating the descriptor file occurs on a non-transactional file system, updates to sets will not be considered an atomic operation. The update to a single object or file will occur immediately (and demonstrate normal transactional behavior). If a failure is encountered for an individual file or object, processing will continue onto the next item in the set and a report will be generated by email to identify the failures.

8. Future Work
The merging and splitting of objects in the DRS requires further definition and may have implications for the DRS 2 Web Admin design that are not yet understood.