A parallel storage and processing server (PS², 1998)

A PS² architecture (Parallel-Storage-and-Processing Server) consists of several storage-and-processing nodes (S/P nodes) each consisting of a small number of processors and disks (Figure 1). An example of PS² architecture is a cluster of 4 BiPentium PCs, each with four storage devices (magnetic disk, CD-ROM, DVD), connected through Fast-Ethernet (100Mb/s). The PS² architecture offers data storage and extraction services to potential users connected through a network. To minimize the network transfers between user and S/P nodes, the processing operations leading to a reduction in data size are performed by the S/P nodes. To maximize performance, disk accesses, data processing and network transfers must be performed in pipeline/parallel fashion.

FIGURE 1. Parallel storage and processing architecture

The difficulty is to coordinate the activity of the S/P-node disks, the S/P-node processors, the network and the user(s) while supporting many kinds of applications (2-D and 3-D imaging, linear algebra, correlation algorithms, ...). The operations to be performed by the S/P nodes are application dependent, and the PS² application developer must be able to specify the scheduling of disk accesses, data transfers and sequencing processing operations required for each application. CAP) is a solution to help the PS² application developer to specify schedulings in a parallel architecture.

FIGURE 2. 2-D slice extraction a 3-D image divided in rectangular extents

A typical application for the PS² architecture is the extraction of a 2-D slice in a 3-D volume such as a CT scan or the Visible Human data set (Figure 2). The 3-D volume (10GB, 5400x2000x1200 pixels) is divided in 32³-pixel extents stored on multiple disks on the PS² architecture. The user specifies the slice to be extracted ; the contributing extents are read from the disks ; slice tiles are then extracted extents by extents on the S/P nodes ; finally, slice tiles are sent to the user and merged on the user computer to generate the complete extracted slice. Performing the slice tile extraction on the S/P node divides the data transferred over the network by 32 : what is transferred over the network is a 32²-pixel 2-D slice tile, rather than a 32³-pixel 3-D extent. All S/P nodes work in parallel. The four operations required to perform the 3-D slice extraction, namely reading an extent, extracting a 2-D slice tile from the extent, transferring the extent from the S/P node to the user computer and merging slice tiles are performed in pipeline for extents stored on the same S/P node. At the same time,

1. a slice tile is being merged,
2. the next slice tile is transferred between the S/P node and the user,
3. the next slice tile is being extracted from an extent,
4. and the next extent is being read from disk.

For more information on the parallel file striping system beneath the PS² architecture, consult the PhD thesis of Vincent Messerli.