This chapter explains how Essbase stores and compresses data on disk.
This chapter includes the following sections:
Essbase uses a data file to store data blocks. By default, a data file is located in its associated database folder. Data files follow the naming convention ESSn.PAG, where n is greater than or equal to one and less than or equal to 65,535.
Essbase uses an index file to store the index for a database. By default, an index file is located in its associated database folder. Index files follow the naming convention ESSn.IND, where n is greater than or equal to one and less than or equal to 65,535.
Essbase automatically allocates storage for data and index files. You can use disk volumes to control how storage is allocated for these files.
To specify disk volumes so that you control how storage is allocated, use this procedure:
You can view index file (.IND file) and data file (.PAG file) names, counts, sizes, and totals, and you can determine whether each file is presently open in Essbase. From Application Manager, select Database > Information and click the Files tab to view this information.
You can also use the Database Properties window and click Storage tab to view this information. For more information, see Essbase Administration Services Online Help.
You can use the LISTFILES command in ESSCMD to view index file and data file names, counts, sizes, and totals and to determine whether each file is presently open in Essbase. See the Technical Reference in the docs directory for information about the command.
Note: The file size information that is provided by the Windows NT operating system for index and data files that reside on NTFS volumes may not be accurate. The file size information provided by Application Manager and by LISTFILES is accurate.
Use disk volumes to specify where you want to store Essbase index files (ESSn.IND) and data files (ESSn.PAG). If you do not use the disk volumes setting, Essbase stores data only on the volume where the ARBORPATH directory resides. If the ARBORPATH variable is not set, Essbase stores data only on the volume where the server was started. For information on setting ARBORPATH, see the Essbase Installation Guide.
Note: For information about checking the size of the index and data files, see Checking Index and Data File Sizes.
You can specify disk volumes using Application Manager (Database > Settings > Storage tab), Administration Services (Storage tab of the Database Properties window), or ESSCMD. When you use disk volumes, Essbase provides the following options for each vo.lume:
Caution: If you specify a volume name but not a volume size, Essbase uses all available space on the volume.
Essbase creates new data files and index files in either of these situations:
For example, suppose you want to use up to 12 GB for Essbase files on volume E, 16 GB on volume F, and 16 GB on volume G. Essbase creates a new file on volume F when the sizes of the index and data files reach 12 GB on volume E and more data needs to be written out to disk.
On volume G, Essbase creates file ess00001.ind and fills it to the default limit of 2 GB. On volume G, Essbase creates file ess00001.pag and fills it to 1 GB.
You have specified a limit of 16 GB on volume G, and you have used 3 GB. You have 13 GB left to use on volume g, but ess00001.ind has reached the maximum file size of 2 GB. The next time Essbase needs storage space when writing index files to disk, Essbase creates a new file on volume G and names it ess00002.ind. Essbase then fills ess00002.ind to its 2 GB limit and creates ess00003.ind. Essbase follows the same procedures for data files.
Figure 503: Example of How Essbase Stores Files Across Volumes
Essbase names files consecutively, starting with ESS00001.xxx, where xxx is IND for an index file and PAG for a data file, and continuing up to ESS65535.xxx. This naming convention applies to each volume, so in the above example, volumes E, F, and G each have files named ESS00001.PAG and ESS00001.IND.
Keep in mind the following guidelines when specifying disk volumes:
To add a new volume or to change values for a previously allocated volume:
Partition Size represents the maximum space that Essbase uses on the volume. The default value is Unlimited; the minimum is 8,192 kilobytes.
You can specify index files, data files, or both. The default is Index + Data.
File Size represents the maximum size that a file specified in File Type can attain before Essbase creates a new file. The default value is 2 GB (2097152 kilobytes); the minimum value is 8,192 kilobytes.
To stop Essbase from storing additional files on an allocated volume, enter the volume name in the Disk Volume text box and click Delete.
Essbase writes no new files to that volume, but continues to access and use files previously created on that volume. For example, suppose you want to stop storing information on Volume F. Select the volume definition for volume F and click Delete, as shown in Figure 504.
Figure 504: De-allocating a Volume
Now, Essbase stores no additional data on Volume F.
To allocate a new volume, enter SETDBSTATEITEM 23 in ESSCMD and either follow the prompts or supply the required values on the command line.
ESSCMD prompts you for the number of new disk volumes you want to add, unless you supply the number on the command line.
Then, for each new volume, ESSCMD prompts you for the following values, unless you supply them on the command line.
When you use ESSCMD, you can specify volume size in bytes (B), kilobytes (K), megabytes (M), gigabytes (G), or terabytes (T). ESSCMD displays minimum, maximum, and current values and 0 for unlimited.
When you use ESSCMD, you can specify file size in bytes (B), kilobytes (K), megabytes (M), or gigabytes (G). ESSCMD displays minimum, maximum, and current values.
The following example allocates up to 10 gigabytes on Volume E, sets a maximum file size of 2 gigabytes, and specifies that data files should be stored only on E:
SETDBSTATEITEM 23 "JILLAPP" "JILLDB" "1" "E" "10G" "2" "2G"
To change the settings on an allocated volume, enter SETDBSTATEITEM 24 in ESSCMD and either follow the prompts or supply the required values on the command line.
ESSCMD prompts you for the following values, unless you supply them on the command line:
The following example allocates up to 20 gigabytes on Volume C and sets a maximum file size of 2 gigabytes:
SETDBSTATEITEM 24 "JILLAPP" "JILLDB" "1" "C" "20G" "3" "2G"
To stop Essbase from storing additional files on a volume, enter SETDBSTATEITEM 25 in ESSCMD and either follow the prompts or supply the required values on the command line. Essbase continues accessing files on the deallocated volume, but does not write new files to it.
ESSCMD prompts you for the following value, unless you supply it on the command line: Delete which volume definition. Use the GETDBSTATE command in ESSCMD to see a list of the currently defined disk volumes and to see the number assigned to each volume.
The following example deallocates the volume that is specified as fourth:
SETDBSTATEITEM 25 "JILLAPP" "JILLDB" "4"
For more syntax information, see the Technical Reference in the docs directory.
On UNIX, volume_name is a mounted UNIX file system. You must enter a fully qualified path name up to the name of the directory you are using for Essbase. Essbase automatically appends the \app directory to the path; you do not specify the \app directory.
Consider the following examples:
/vol2/essbase 10M
Volume size is the maximum space, in kilobytes, allocated to the volume. The default value is unlimited-Essbase uses all available space on that volume.
Assume you want to use up to 20 GB for Essbase files on Volume E, 25 GB on Volume F, and 25 GB on Volume G. You are using the default file size limit of 2 GB.
When you load data, Essbase stores up to 20 GB on Volume E; if the database is larger than 20 GB, Essbase stores the next 25 GB on Volume F, and so on.
Figure 505: Example of Using Disk Volumes
Essbase allows you to choose whether or not data blocks that are stored on disk are compressed, as well as which compression scheme to use. When data compression is enabled, Essbase compresses data blocks when it writes them out to disk. Essbase fully expands the compressed data blocks, including empty cells, when the blocks are swapped into the data cache.
Generally, data compression optimizes storage use. You can check compression efficiency by checking the compression ratio statistic. See Checking the Compression Ratio for more information.
Essbase provides two options for data compression:
Because Essbase compresses data blocks as they are written to disk, it is possible for bitmap, RLE, and non-compressed data blocks to coexist in the same data file. Keep in mind the following:
You may want to disable data compression if blocks have very high density (90% or greater) and have few consecutive, repeating data values. Under these conditions, enabling compression consumes resources unnecessarily.
With bitmap compression, Essbase uses a bitmap to represent data cells, and stores only the bitmap, the block header, and the other control information. A bitmap uses one bit for each cell in the data block, whether the cell value is missing or non-missing. When a data block is not compressed, Essbase uses 8 bytes to store every non-missing cell.
When using bitmap compression, Essbase stores only non-missing values and does not compress repetitive values or zeros (contrast with RLE compression, described in RLE Data Compression). When Essbase pages a data block into the data cache, it fully expands the data block, using the bitmap to recreate the missing values.
Because the bitmap uses one bit for each cell in the data block, the bitmap scheme provides a fixed overhead for data compression. Figure 506 represents a portion of a data block, as an example. In this example, Essbase uses 64 bytes to store the data in the fully expanded block, but uses one byte (eight bits) to store the bitmap of the compressed data on disk. (Essbase also uses a 72-byte block header for each block, whether the block is compressed or not.)
Figure 506: Bitmap Data Compression
In most cases, bitmap compression conserves disk space more efficiently. However, much depends on the configuration of the data.
When using the run-length encoding (RLE) compression scheme, Essbase compresses any consecutive, repetitive values-any value that repeats three or more times consecutively, including zero. Essbase keeps track of each repeating value and the number of times it is repeated consecutively.
In the example in Figure 507, Essbase uses 64 bytes to store the data in the fully expanded block, but uses 56 bytes to store the compressed data on disk. (Essbase also uses a 72-byte block header for each block, whether the block is compressed or not.)
Figure 507: RLE Data Compression
The RLE compression type may be preferable under the following circumstances:
In Application Manager and Administration Services, data compression is enabled by default. You can view the current settings by selecting Database > Settings and clicking the Storage tab or by opening the Database Properties window and clicking the Storage tab.
Changes to the data compression setting take effect immediately as Essbase writes data blocks to disk. For blocks already on disk, Essbase does not change compression schemes or enable or disable compression. When you change the data compression settings of blocks already on disk, Essbase uses the new compression scheme the next time Essbase accesses, updates, and stores the blocks.
You can enable or disable data compression by using SETDBSTATEITEM or SETDBSTATE. To set the data compression type, you must use SETDBSTATEITEM.
To enable or disable data compression, enter SETDBSTATEITEM 14 in ESSCMD and either follow the prompts or supply the required values on the command line.
ESSCMD prompts you for the following values, unless you supply them on the command line:
To specify the data compression type, enter SETDBSTATEITEM 15 in ESSCMD and either follow the prompts or supply the required values on the command line. ESSCMD prompts you for a value of "1" (run length encoding) or "2" (bitmap, the default).
The following example enables Bitmap compression:
SETDBSTATEITEM 14 "JILLAPP" "JILLDB" "Y" "2"
For more syntax information, see the Technical Reference in the docs directory.
To set the data compression type, you must use SETDBSTATEITEM or the Database Settings dialog box.
For more syntax information, see the Technical Reference in the docs directory.
The compression ratio represents the ratio of the compressed block size (including overhead) to the uncompressed block size, regardless of the compression type in effect. Overhead is the space required by mechanisms that manage compression/expansion.
In Application Manager, select Database > Information and click the Statistics tab to check the compression ratio.
Tip: You can also check the compression ratio without using the Application Manager.
Tool |
Instructions |
For more information |
---|---|---|
Note: The larger the number, the more compression. The compression ratio can vary widely from block to block.
Data block size is determined by the amount of data in a particular combination of dense dimensions. For example, when you change the dense or sparse configuration of one or more dimensions in the database, the data block size changes. Data block size is 8n bytes, where n is the number of cells that exist for that combination of dense dimensions.
Note: Eight to 100 kilobytes is the optimum size range.
For information about determining the size of a data block, see Size of Expanded Data Block.
Tip: You can use any of the following tools to view the block size for a database.
Tool |
Instructions |
For more information |
---|---|---|
Select Database > Information and click the Statistics tab to see the value: block size in bytes. |
||
![]() © 2002 Hyperion Solutions Corporation. All rights reserved. http://www.hyperion.com |