gmSCPagingSystemClass

The PagingSystem Service Class

The service class PagingSystem manages the access of files whose entire content may not be stored in memory at the same time. These files are ultimately read and written via the Offline class, but they contain information that must be persistent which either grows during the time that it is being used or is too large to be stored entirely in memory. To work with these files, they are broken into fixed sized blocks, not all of which need to be memory resident at the same time. The methods in this class keep track of which blocks are currently in memory, where those blocks are in memory, and whether or not those blocks have been changed since they were last accessed. In addition, they must decide which file block to remove from memory when the allocated memory block count for the file has been exhausted.

There are three different approaches that can be used — BlockVector, BlockSequence, and SearchForBlock. The BlockVector approach keeps a linear byte array indexed by the maximum number of blocks to be stored on the file. Each byte simply contains the number of the memory area containing the file block, if that block is in memory, or a zero if that block is not in memory. This approach is by far the fastest, but it requires as many bytes of memory as there are blocks on the file and it requires that the maximum size of the file be known when it is opened or created. The BlockSequence approach uses the MemorySequence class to maintain the memory block numbers that correspond to the file block numbers. The advantage here is that the maximum size of the file need not be known when it is opened or created. The disadvantage is slightly slower performance and the need to potentially create an extremely large memory sequence. The SearchForBlock approach keeps a linear array indexed by the maximum number of blocks allowed in memory. The blocks are always kept in numeric order by their number in the file. Each time a block is accessed, this array is binary searched to determine if the block is currently in memory. This approach is the most general since its memory requirements are independent of the size of the file but this approach is also the slowest.

The names of these paging approaches are in the PagingType enumeration. This enumeration contains two additional entries — MemoryMapped and MemorySequence. These two approaches retain the entire file content in memory and can obviously only be used when the file size makes this possible. They are both much more efficient then approaches describe above, but do not support persistence. The MemoryMapped approach has the entire file mapped into memory. It can be used to work with a previously created file whose size will not change. The MemorySequence approach stores block pointers in a MemorySequence and therefore allows the information stored to grow; however, there is no actual file backing the information up so it is not persistent.

The notion of semi-permanent pointers pervades this approach to information management. The basic idea is that a page pointer is a valid memory pointer until the block which it contains is swapped out of memory. Thus these pointers can be used safely within local blocks of code where efficiency is needed, even though they cannot be saved and reused over longer stretches of code.

The method PagingSystem_Access

Prototype

UBYTE* PagingSystem_Access(void* This,int BlockNumber,int Delta);

The PagingSystem_Access method accesses a block from a paged file. It returns a pointer to the start of a page in the paging system which contains a specified block from the file being managed. Blocks are numbered sequentially starting at 1. Note that passing the method a block number larger than the current block count may force it to add new blocks to the back of the file. This method should only be passed existing block numbers. The method PagingSystem_AddBlock should be used to expand the file. Its parameters are:

Parameter	Description
This	Specifies the handle for the paging system to be accessed. This must have been obtained from the PagingSystem_Create method.
BlockNumber	Specifies the number of the desired block, where blocks are numbered starting at 1, for the desired page.
Delta	Specifies a flag, which if non-zero, indicates that the caller intends to change the information within the block. Only blocks accessed via a non-zero delta are written back to their file when the block is removed from memory.

The method returns a semi-permanent pointer to the start of the block, or a NULL if something went wrong.

The method PagingSystem_AddBlock

Prototype

UBYTE* PagingSystem_AddBlock(void* This,int* BlockNumber);

The PagingSystem_AddBlock method adds a new block to a paging system and returns a semi-permanent pointer to its memory image. In addition, the number of the block may be returned. Note that this method is the only one to be used to add blocks to the file. The PagingSystemAccess method is only used to access existing blocks. Its parameters are:

Parameter	Description
This	Specifies the handle for the paging system to be accessed. This must have been obtained from the PagingSystem_Create method.
BlockNumber	returns the number of the block that been has created, if it is not NULL.

If the paging system has a maximum number of blocks specified, then it returns NULL if that maximum has been exceeded; else it returns a semi-permanent byte pointer to the start of the new block. This new block is initialized at zero.

The method PagingSystem_Close

Prototype

void PagingSystem_Close(void* This);

The PagingSystem_Close method closes a PagingSystem instance, by returning any memory occupied by it to the operating system. Before doing so it writes any of its blocks that have been changed in memory to the file being controlled by the system. Note that this method does not close the actual file, just as the PagingSystem_Create method does not open the file. Its parameter is:

Parameter	Description
This	Specifies the handle for the paging system to be accessed. This must have been obtained from the PagingSystem_Create method.

The method has no return value.

The method PagingSystem_Create

Prototype

void* PagingSystem_Create(void* File,int Algorithm,int PageSize,int MaxPages,int MaxBlocks,int BlockCount);

The PagingSystem_Create method creates a PagingSystem instance for a file. This instance can then be used to access blocks on the file as stored in memory pages. Its parameters are:

Parameter	Description
File	Specifies the handle for the file as returned either by the Offline_OpenFile or Offline_CreateFile methods. The methods in the Offline class perform all physical file accesses for the file.
Algorithm	Specifies the algorithm to be used for determining if a given block on the file is currently in memory and if so which page is it in. One of the three enumeration entries: PagingType_BlockVector, PagingType_BlockSequence, or PagingType_SearchForBlock must be used. These are explained in the header for this class.
PageSize	Specifies the size of each page and thus block of the file specified in bytes. If set to zero then the PagingSystem_PageSize value is used. Note that the page size of the file when it is opened must match the pagesize when it is created. It is up to the user of this class to ensure this.
MaxPages	Specifies the maximum number of pages to be stored in memory for the file. This value cannot exceed 255. If it is larger than 255 or zero, then the MAX_PAGES_DEFAULT value is used.
MaxBlocks	Specifies the maximum number of blocks to be on the file. This value is needed only for the BlockVector algorithm. It is used by the other algorithms only by the PagingSystem_AddBlock method as a constraint on the overall filesize. If no such constraint is desired, then set this parameter to zero.
BlockCount	specifies the actual number of blocks on the file. When the file is initially created, this should be zero. When the file is opened it should reflect the actual number of blocks on the file. As with pagesize, it is up to the user of this method to determine this value.

If all goes well this method returns a handle for the control structure that will page the blocks of the file. If there is insufficient memory to allocate the various needed components, then a NULL is returned.