gmSCLongMemoryClass
The LongMemory Service Class
Section titled “The LongMemory Service Class”The service class LongMemory manages long memory areas. In the area of information management program storage must be viewed as containing a wide variety of different structures, indexes, lists, fixed records, long unformatted records, etc.. In addition it may be purely memory bound or it may be stored in a persistent file system. The role of this class is to provide a single interface for storing and retrieving information. It is sufficiently robust to allow for the management of large files in the multiple gigabyte size range; while it is still efficient for use by purely memory bound applications.
Physically, a long memory storage area consists of a series or relatively large fixed sized blocks. The manner in which these blocks are created, maintained, and accessed is determined when the storage area is created. See the discussion within the LongMemory_Create method for a description of the different approaches used. Within these large fixed sized blocks there are four fundamentally different record types stored within the storage area:
- Control Tables — These are relatively short tables that are always stored entirely within a single block. The method LongMemory_Allocate is used to enter new control tables into the storage area.
- Data Tables — These are arbitrarily long vectors that may cross block boundaries, but whose entire expanse consists of a logically contiguous area. The method LongMemory_Expand is used to enter new data tables into the storage area.
- Blocked Tables — These are arbitrary collections of information that use entire storage blocks that need not be contiguous. The manner in which the blocks are used is dependent on the type of information collections. Most complex entities like indexes, storage based property sequences, short volatile records, etc. are blocked tables. The method LongMemory_EmptyBlock is used to allocate blocked tables in the storage area.
- Posting blocks — These are fixed sized linked smaller blocks used to contain posting, collections of integer value such as selection vectors, inverted indexes, and sorting control tables.
At a logical level the storage area is viewed as consisting of a simple sequence of words. In this implementation a word is a 32-bit signed integer. All information is stored on a word boundary and all offsets are expressed in terms of words.
The method LongMemory_Access
Section titled “The method LongMemory_Access”Prototype
int* LongMemory_Access(void* This,int Offset,int Delta);The LongMemory_Access method is the lowest level logical access method in the storage system. The caller specifies a word offset into the storage area that he wishes to access along with his intent to change it. This method returns a semipermanent pointer to that word in the storage area. This method has the logical view of the storage area — i.e., a long linear sequence of words numbered from zero. Its parameters are:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| Offset | Specifies the sequence number relative to zero of the word in the storage area that the user wishes to access. |
| Delta | Specifies, if nonzero, the callers intent to change the content of the word; else he wishes only to read it. |
If all goes well this method returns a semipermanent pointer to the word; else it takes a fatal error branch and does not return.
The method LongMemory_Allocate
Section titled “The method LongMemory_Allocate”Prototype
int LongMemory_Allocate(void* This,int Size);The LongMemory_Allocate method allocates a contiguous area of storage. Though normal “data” storage in the persistent storage area makes no assumptions about the occurrence of block boundaries, the basic entry tables around which the object identifiers and property vectors are built assume that they do not contain any internal boundaries. It is this assumption that makes it possible to use semipermanent pointers to access these areas. This method allocates a stretch of continuous storage in the storage area that does not cross a block boundary. Obviously the length of that area must be less than the pagesize being used in the underlying physical storage of the blocks. Its parameters are:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| Size | Specifies the size in words of the structure being allocated. |
This method returns the actual starting offset in the persistent storage area of the allocated area.
The method LongMemory_AllocateBytes
Section titled “The method LongMemory_AllocateBytes”Prototype
int LongMemory_AllocateBytes(void* This,int Size);The LongMemory_AllocateBytes method allocates sufficient storage to accommodate a specified number of bytes that crosses no block boundaries. The storage starts on a word boundary. Though normal “data” storage in the persistent storage area makes no assumptions about the occurrence of block boundaries, the basic entry tables around which the object identifiers and property vectors are built assume that they do not contain any internal boundaries. It is this assumption that makes it possible to use semipermanent pointers to access these areas. This method allocates a stretch of continuous storage in the storage area that does not cross a block boundary. Obviously the length of that area must be less than the pagesize being used in the underlying physical storage of the blocks. Its parameters are:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| Size | Specifies size in bytes of the structure being allocated. |
This method returns the actual starting offset in the persistent storage area of the allocated area.
The method LongMemory_Close
Section titled “The method LongMemory_Close”Prototype
void LongMemory_Close(void* This);The LongMemory_Close method closes the specified storage area and returns all of its memory resources to the operating system. Its parameter is:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. This must have been obtained from a storage area creation or open method. |
This method has no return value.
The method LongMemory_Create
Section titled “The method LongMemory_Create”Prototype
void* LongMemory_Create(char* FileName,int PagingMethod,int PageSize,int MaximumPages,int MaximumBlocks);The LongMemory_Create method creates a storage area. Once the information about how the blocks of the persistent storage area are to be managed in memory is known, this method creates that area along with the machinery needed to manage its fixed-sized blocks and block boundaries. Note that this is a very low level method. Typically the methods LongMemory_OpenArea or LongMemory_OpenRaw should be used, as these select the best approach to be used given the context of use. Its parameters are:
| Parameter | Description |
|---|---|
| FileName | Contains the name of the file to be used if the persistent storage area is to reside within the file system of the host platform. Note that non-file resident persistent areas must use a the memory mapped paging algorithm. This parameter is NULL for non-file-based persistent storage areas. |
| PagingMethod | Specifies the paging algorithm to be used. One of the following constants must be used: PagingType_BlockVector, PagingType_BlockSequence, PagingType_SarchForBlock, PagingType_MemoryMapped, or PagingType_MemorySequence. |
| PageSize | Specifies the size of each page in the storage area. If set to zero, the value PagingSystem_PageSize is used. Note this value is expressed in bytes not words. |
| MaximumPages | Specifies the maximum number of pages in memory to be allocated for the storage area. |
| MaximumBlocks | specifies the maximum number of blocks to be allocated for the storage area in its file-based form. |
If all goes well, this method returns the storage area handle to be used for the created area; else it returns NULL.
The method LongMemory_DeleteBlock
Section titled “The method LongMemory_DeleteBlock”Prototype
void LongMemory_DeleteBlock(void* This,int BlockNumber);The LongMemory_DeleteBlock method logically deletes a block from blocked-table storage. The block is not physically deleted, rather it is entered into the deleted block chain. From there it can be reused via the LongMemory_EmptyBlock or the LongMemory_Allocate methods. Its parameters are:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| BlockNumber | contains the number of the block to be marked as being deleted. |
This method has no return value.
The method LongMemory_DeleteRecord
Section titled “The method LongMemory_DeleteRecord”Prototype
void LongMemory_DeleteRecord(void* This,int ioiad,int length);The LongMemory_DeleteRecord method logically deletes a contiguous record from long memory. The record is not physically deleted, rather it is posted into a deleted record chain. From there it can be reused via the LongMemory_Allocate method. Its parameters are:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| ioiad | Specifies the starting address of the record in the long memory area. |
| length S | pecifies the length of the contiguous area being deleted. |
This method has no return value.
The method LongMemory_EmptyBlock
Section titled “The method LongMemory_EmptyBlock”Prototype
int* LongMemory_EmptyBlock(void* This,int* BlockNumber);The LongMemory_EmptyBlock method gets an empty block from a storage area. It is used whenever a block is needed by a blocked-table storage user. The block may be at the end of the storage area or it may be a block somewhere inside of the area that was marked as being deleted by the method LongMemory_DeleteBlock. To emphasize, blocks returned by this method are not guaranteed to be contiguous. If contiguous long storage is needed use the LongMemory_Expand method. Its parameters are:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| BlockNumber | returns the block number of the empty block. It is this block number that is the primary persistent retrieval value for the block in all future accesses. |
If all goes well this method returns a semipermanent pointer to the start of the block; else it returns a NULL and stores a zero in the BlockNumber parameter.
The method LongMemory_Extend
Section titled “The method LongMemory_Extend”Prototype
int LongMemory_Extend(void* This,int nWord);The LongMemory_Extend method extends a storage area. Information can only be put or gotten from areas that have been previously allocated either via the LongMemory_Allocate or the LongMemory_Extend methods. This method extends the persistent storage area by the indicated number of words without regard to any block boundaries that might be spanned by the new area. This area then can be used for data storage but not for control tables and property vector storage. Its parameters are:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| nWord | Specifies the number of words to extend the storage area by. |
This method returns the actual offset in the persistent storage area of the start of the new area.
The method LongMemory_GetBlock
Section titled “The method LongMemory_GetBlock”Prototype
int* LongMemory_GetBlock(void* This,int BlockNumber,int Delta);The LongMemory_GetBlock method is the lowest level physical access method for the persistent storage area. The caller specifies a physical block number that he wishes to access along with his intent to change it. This method returns a semipermanent pointer to the start of the block in memory. Its parameters are:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. This must have been obtained from a storage area creation or open method. |
| BlockNumber | Specifies the number of the block to access. |
| Delta | Specifies the callers intent to change the content of the block. If it is nonzero, then the user intends to change the content of the block; else he will only read it. |
If all goes well this method returns a semipermanent pointer to the start of the block, else it takes a fatal error branch and does not return.
The method LongMemory_GetDeleted
Section titled “The method LongMemory_GetDeleted”Prototype
int LongMemory_GetDeleted(void* This,int nAllocate);The LongMemory_GetDeleted method determines if a deleted contiguous record exists that is at least a specified size long. If so it removes it from that deleted record chain and returns its offset in the long memory area. This method is used by the contiguous record allocation methods before they attempt to allocate new memory to satisfy a request. Its parameters are:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. This must have been obtained from a storage area creation or open method. |
| nAllocate | specifies the length of the desired allocation in words. |
If a deleted segment is found that contains sufficient room to satisfy the request, then its offset in the long memory area is returned. Else a zero is returned.
The method LongMemory_GetNext
Section titled “The method LongMemory_GetNext”Prototype
int LongMemory_GetNext(void* This);The LongMemory_GetNext method supplies the retrieval offset associated with the next available word of long memory. The user may then begin writing to this storage. Note that the user is guaranteed that he can perform offset arithmetic with this retrieval value however, he is not guaranteed that the memory associated with these values is contiguous. Thus information may only be copied into (written) and copied from (read) the storage area. Its parameter is:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
This method returns the retrieval offset for the next available word of storage.
The method LongMemory_GetPageSize
Section titled “The method LongMemory_GetPageSize”Prototype
int LongMemory_GetPageSize(void* This);The LongMemory_GetPageSize method is for use by those storage applications that use their own internal block layouts. Its parameter is:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
This method returns the page size of the storage area.
The method LongMemory_GetPostingSize
Section titled “The method LongMemory_GetPostingSize”Prototype
int LongMemory_GetPostingSize(void* This);The LongMemory_GetPostingSize method returns the posting size of the storage area for use by those storage applications that use their own internal block layouts. Its parameter is:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
This method returns the posting size of the storage area.
The method LongMemory_HasChanged
Section titled “The method LongMemory_HasChanged”Prototype
int LongMemory_HasChanged(void* This);The LongMemory_HasChanged method specifies if any changes have been made to the storage area. If not, then the closing of various components can either be skipped or simplified. Its parameter is:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
If the storage area has not been changed, then a zero is returned. If it has been changed, then a nonzero value is returned.
The method LongMemory_Open
Section titled “The method LongMemory_Open”Prototype
void* LongMemory_Open(char* FileName,int PagingMethod,int AccessType,int MaximumPages,int MaximumBlocks);The LongMemory_Open method opens a storage area. Once the information about how the blocks of a previously created persistent storage area are to be managed in memory is known, this method opens that storage area and creates the machinery needed to manage its fixed-sized blocks and block boundaries. Note that this is a very low level method. Typically the method LongMemory_OpenArea or LongMemory_OpenRaw should be used, as these select the best approach to be used given the context of use. Its parameters are:
| Parameter | Description |
|---|---|
| FileName | Contains the name of the file used for the persistent storage area when it was created. |
| PagingMethod | Specifies the paging algorithm to be used. One of the following constants must be used: PagingType_BlockVector, PagingType_BlockSequence, PagingType_SearchForBlock, PagingType_MemoryMapped, or PagingType_MemorySequence. |
| AccessType | Specifies if access is be read-write, value is zero, or one if it is to be readonly. |
| MaximumPages | Specifies the maximum number of pages in memory to be allocated for the storage area. |
| MaximumBlocks | specifies the maximum number of blocks to be allocated for the storage area in its file-based form. If zero then the number of blocks currently on the file is used. |
If all goes well,this method returns the storage area handle to be used to reference the storage area. A NULL return value means that the file could not be opened.
The method LongMemory_NewBlock
Section titled “The method LongMemory_NewBlock”Prototype
int* LongMemory_NewBlock(void* This);The LongMemory_NewBlock method adds a new block to the back of the storage area. It is used in those cases where long sequential records are being added to the memory area which must span multiple sequential blocks. It is of course also used by the various allocation methods that require storage space when no deleted blocks are available. Its parameter is:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
If no new blocks can be allocated to the long memory area then a NULL is returned; else a semipermanent pointer to the start of the block is returned.
The method LongMemory_OpenArea
Section titled “The method LongMemory_OpenArea”Prototype
void* LongMemory_OpenArea(char* FileName,int Mode,int ReadOnly);The LongMemory_OpenArea method creates or opens a long storage area. It selects the best paging method depending upon the context of use. Its parameters are as follows:
| Parameter | Description |
|---|---|
| FileName | Contains the name of the file used for the long storage area when it was created or the name to be used if it is to be a new storage area. |
| Mode | Specifies the mode of the file and must be one of the following constants — LongMemory_Open_Existing, LongMemory_Open_New, LongMemory_Open_Virtual, or LongMemory_Open_Memory. |
| ReadOnly | Specifies the access type. It contains zero if access is the be read-write and one if it is to be readonly. |
If all goes well,this method returns the storage area handle to be used to reference the storage area else it returns a NULL.
The method LongMemory_OpenRaw
Section titled “The method LongMemory_OpenRaw”Prototype
void* LongMemory_OpenRaw(char* FileName,int Mode,int ReadOnly);The LongMemory_OpenRaw method creates or opens an existing storage area that has it own internal structure. No boot record is maintained and no deleted block or segment information is retained. The only methods that should be used with raw storage areas are: LongMemory_Access, LongMemory_Close, LongMemory_Read, LongMemory_Rewrite, or LongMemory_Write. Its parameters are:
| Parameter | Description |
|---|---|
| FileName | Contains the name of the file used for the persistent storage area when it was created. |
| Mode | Specifies the mode of the file and must be one of the following constants — LongMemory_Open_Existing, LongMemory_Open_New, LongMemory_Open_Virtual, or LongMemory_Open_Memory. |
| ReadOnly | Specifies the access type. It contains zero if access is the be read-write and one if it is to be readonly. |
If all goes well this method returns the handle for the controlling storage structure; else it returns a NULL.
The method LongMemory_ReadBytes
Section titled “The method LongMemory_ReadBytes”Prototype
void LongMemory_ReadBytes(void* This,int Offset,UBYTE* Bytes,int nByte);The LongMemory_ReadBytes method reads bytes of information from persistent storage by copying them into a buffer supplied to this method. A byte-stream is retrieved using this method. Doing the copy ensures that the ultimate user of these bytes need not be concerned with boundary problems and need not be concerned over the duration of use of those values. This method has the logical view of the storage area — i.e., a long linear sequence of words numbered from zero. Its parameters are as follows:
| Parameter | Description |
|---|---|
| This | Specifies the hhandle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| Offset | Specifies the word offset of the start of the bytes to be read. |
| Bytes | Receives the bytes read. |
| nByte | Specifies the number of bytes to be read. |
The method LongMemory_RewriteBytes
Section titled “The method LongMemory_RewriteBytes”Prototype
void LongMemory_RewriteBytes(void* This,int Offset,UBYTE* Bytes,int nByte);The LongMemory_RewriteBytes method rewrites bytes of information into persistent storage by copying them from a buffer supplied to this method. This method is used for storing byte-streams as opposed to word streams. Its parameters are as follows:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| Offset | Specifies the word offset of the start of the bytes to be written. |
| Bytes | Contains the bytes to be rewritten. |
| nByte | Specifies the number of bytes to be rewritten. |
The method LongMemory_ZeroBytes
Section titled “The method LongMemory_ZeroBytes”Prototype
void LongMemory_ZeroBytes(void* This,int Offset,int nByte);The LongMemory_ZeroBytes method zeros bytes of information in persistent storage. This method is used for initializing byte-streams as opposed to word streams. Its parameters are:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| Offset | Specifies the word offset of the start of the bytes to be zeroed. |
| nByte | Specifies the number of bytes to be zeroed. |
The method LongMemory_WriteBytes
Section titled “The method LongMemory_WriteBytes”Prototype
int LongMemory_WriteBytes(void* This,UBYTE* Bytes,int nByte,int nStart);The LongMemory_WriteBytes method writes byte-stream information to the back of the storage area. Its parameters are as follows:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| Bytes | Contains the bytes to be written. |
| nByte | Specifies the number of bytes to be written. |
| nStart | Specifies the number of bytes that must be written as part of a contiguous block — i.e., without an intervening block boundary. |
The return value from the method is the actual offset in persistent storage of the start of the new area. Note that the offset is word-oriented, not byte oriented.
The method LongMemory_Read
Section titled “The method LongMemory_Read”Prototype
void LongMemory_Read(void* This,int Offset,int* Buffer,int nWord);The LongMemory_Read method reads words of information from persistent storage by copying them into a buffer supplied to this method. Most “data” is retrieved using this method. Doing the copy ensures that the ultimate user of these words need not be concerned with boundary problems and need not be concerned over the duration of use of those values. This method has the logical view of the storage area — i.e., a long linear sequence of words numbered from zero. Its parameters are as follows:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| Offset | Specifies the word offset of the start of the words to be read. |
| Buffer | Receives the words read. |
| nWord | Specifies the number of words to be read. |
The method LongMemory_Rewrite
Section titled “The method LongMemory_Rewrite”Prototype
void LongMemory_Rewrite(void* This,int Offset,int* Buffer,int nWord);The LongMemory_Rewrite method rewrites words of information in persistent storage by copying them from a buffer supplied to this method. Most “data” is stored using this method. This is because the normal process of adding records to the back of storage involves extending it first and then doing a rewrite into the extension. Doing the copy ensures that the ultimate creator of these words need not be concerned with boundary problems. Its parameters are as follows:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| Offset | Specifies the offset of the start of the words to be rewritten. |
| Buffer | Contains the words being rewritten. |
| nWord | Specifies the number of words to be rewritten. |
The method LongMemory_Write
Section titled “The method LongMemory_Write”Prototype
int LongMemory_Write(void* This,int* Record,int nRecord);The LongMemory_Write method writes words to back of a storage area. Its parameters are as follows:
| Parameter | Description |
|---|---|
| This | Specifies the handle for the storage area control structure. It must have been obtained from a storage area creation or open method. |
| Record | Contains the words to be written. |
| nRecord | Specifies the number of words to be written. |
The method returns the actual offset in storage of the start of the record. The record can be later read using this offset.