This chapter describes the memory caches that Essbase uses and provides recommendations for cache-related settings.
This chapter includes the following sections:
Essbase uses five memory caches to coordinate memory usage:
Essbase provides default size settings for each cache. You can adjust the size of any of these five caches as needed for each database. Appropriate cache size is affected by many factors, including database size, block size, index size, and available memory on the server. Cache size settings can effect database and general server performance significantly.
Use these topics to properly size caches for performance:
Before setting cache sizes, you need to choose to enable cache memory locking or leave the default, that cache memory locking is disabled.
The setting for cache memory locking controls whether the memory used for the index cache, data file cache, and data cache is locked into physical memory, giving the Essbase kernel priority use of system RAM.
To use cache memory locking, you must be using direct I/O (buffered I/O is the default I/O access mode), and direct I/O requires a larger index cache size than buffered I/O. For more information, see "Migrating and Upgrading Databases" in the Essbase Installation Guide, and Managing Essbase Kernel Settings.
Locking improves performance for a Essbase database because the system memory manager does not need to swap the memory used by the caches when swapping the memory used by OLAP Server. By default, cache memory locking is turned off.
Enabling cache memory locking gives the Essbase Kernel priority use of system RAM. If you enable cache memory locking, leave at least one-third of the system RAM available for non-Essbase Kernel use. If you do not want to give the Essbase Kernel priority usage of system RAM, do not enable cache memory locking.
To enable cache memory locking, use this procedure:
Figure 536: Database Settings Window-Changing Cache Memory Locking
Note: You must restart the database to initialize the new cache memory locking setting.
Tip: You can enable memory cache locking without Essbase Application Manager:
Tool |
Instructions |
For More Information |
---|---|---|
The settings that you should use for each of the caches that you can configure depends on data distribution and the dense/sparse configuration of the database.
If memory resources are restricted, you can optimize performance by adjusting the cache settings relative to the memory available on the machine which contains your database.
The needs for each site and even for a particular database can vary. Depending on the complexity and type of each operation, Essbase allocates as much memory for the data file cache and the data cache as needed. Use the recommended values in this section to estimate enough memory for optimal performance.
If you are using Essbase for the first time, cache sizes are automatically set to the default values discussed in the following sections. If you are migrating from Essbase Release 5.x, the data file cache is set to the default value and the other cache settings from that version are retained when you migrate. See the Essbase Installation Guide for migration information.
Note: Changes made to cache sizes take effect the next time you start the database.
Use these topics to find and understand recommendations for each cache size:
Note: The size of index pages is fixed at 8 K. This is to reduce input-output overhead, as well as to simplify database migration.
The index is stored in index files on disk. When a database is active, the most recently accessed index pages are held in the index cache, which is a memory area that is allocated for index pages. How much of the index can be held in memory at one time depends upon the amount of memory you allocate to the index cache.
When a data block is requested, Essbase looks at the index pages in the index cache to find its location on disk. If the block location is not found in index pages in the index cache, the index page containing the block location is pulled into the index cache from disk. If the index cache is full, the least recently used index page in the cache is dropped to make room for the index page containing the location of the data block.
The effectiveness of the index cache size depends on the nature of the calculation you are performing. For example, if you were reloading and recalculating an entire database (such as a database that is refreshed each month), a high index cache size is not helpful because Essbase is creating new blocks rather than searching the index cache for existing blocks during calculation.
Table 79 shows default and recommended settings for the index cache.
Minimum Value |
Default Value |
Recommended Value |
---|---|---|
Combined size of all ESSn.IND files, if possible; as large as possible otherwise. Do not set this cache size higher than the total index size, as no performance improvement results. (To determine the total index size, see Index Files.) |
In general, if you are using direct I/O, make the index cache as large as system resources allow, up to 2 GB. If you are using buffered I/O, make them as small as possible.
For information on testing and fine tuning your cache settings, see Fine Tuning Cache Settings.
To set the size of the index cache:
Figure 537: Database Settings Window-Changing Index Cache Size
Tip: You can change the index cache size without Application Manager:
Tool |
Instructions |
For More Information |
---|---|---|
The data file cache holds data files (.PAG files) in memory, if you are using direct I/O. If you are not using direct I/O, the data file cache is not used. How much of the data within data files can fit into memory at one time depends on the amount of memory you allocate to the data file cache.
In general, if you have to choose whether to allocate memory to the data cache or to the data file cache, choose the data file cache if you are using direct I/O.
Table 80 shows default and recommended settings for the data file cache.
Minimum Value |
Default Value |
Recommended Value |
---|---|---|
Combined size of all ESSn.PAG files, if possible; otherwise as large as possible. This cache setting not used if Essbase is set to use buffered I/O. |
In general, if you are using direct I/O, make the data file cache as large as system resources allow, up to 2 GB. If you are using buffered I/O, the data file cache is not used.
For information on testing and fine tuning your cache settings, see Fine Tuning Cache Settings.
To set the size of the data file cache:
Figure 538: Database Settings Window-Changing the Data File Cache Size
Tip: You can change the data file size without Application Manager:
Tool |
Instructions |
For More Information |
---|---|---|
Data blocks reside on physical disk and in memory. The data cache is the memory area that is allocated to hold uncompressed data blocks. The number of blocks that can be held in the data cache at one time depends on the amount of memory you allocate to the data cache.
When a block is requested, Essbase searches the data cache for the block. If Essbase finds the block in the cache, it is accessed immediately. If the block is not found in the cache, Essbase searches the index for the appropriate block number and then uses the index entry of the block to retrieve it from the appropriate data file on disk. Retrieving a requested block from the data cache is faster, and therefore improves performance.
In general, if you have to choose whether to allocate memory to the data cache or to the data file cache, choose the data file cache if you are using direct I/O.
This table shows default and recommended settings for the data cache.
Minimum Value |
Default Value |
Recommended Value |
---|---|---|
0.125 * the value of data file cache size. Increase value if either of these conditions exist:
|
Make the data cache as small as possible whether you are using buffered I/O or direct I/O.
For information on testing and fine tuning cache settings, see Fine Tuning Cache Settings.
To set the size of the data cache, use this procedure:
Figure 539: Database Settings Window-Changing the Data Cache Size
Tip: You can change the data cache size without Application Manager:
Tool |
Instructions |
For More Information |
---|---|---|
Essbase can create a bitmap, whose size is controlled by the size of the calculator cache, to record and track data blocks during a calculation. Determining which blocks exist using the bitmap is faster than accessing the disk to obtain the information, particularly if calculating a database for the first time or calculating a database when the data is very sparse.
Essbase uses the calculator cache bitmap if the database has at least two sparse dimensions, and either of these conditions are also met:
The best size for the calculator cache depends on the number and density of the sparse dimensions in your outline. Use these topics to understand the calculator cache bitmap, size the calculator cache, and change the size of the calculator cache (and therefore the largest possible size for the bitmap), if required:
For the calculator cache, Essbase separates sparse dimensions in the database into two groups:
Essbase starts with the first sparse dimension in the database outline and fits as many sparse dimensions as possible into the bitmap. The dimensions that fit are the bitmap dimensions. Essbase stops the process when it cannot fit another complete sparse dimension into the bitmap. Because the calculator cache controls the size of the bitmap, the number of sparse dimensions that can fit in the bitmap depends on the size of the calculator cache (and the number and size of the sparse dimensions).
The remaining sparse dimensions are the anchoring dimensions. For anchoring dimensions, Essbase cannot use the bitmap and must to determine whether or not blocks exist.
To see which dimensions are anchoring dimensions and which are bitmap dimensions, use the SET MSG DETAIL calculation command to display bitmap information in the application log. For more information on this command, see the Technical Reference in the docs directory.
Carefully order the sparse dimensions in your outline so that as many dimensions as possible can be placed into the bitmap. Start with the dimension that contains the fewest members, and continue until the dimension with the most members is last. This order allows more dimensions to fit into the bitmap and results in improved calculation performance.
Note: The order of sparse dimensions in the outline also affects query performance. To optimize the outline for query performance, see Designing an Outline to Optimize Performance.
Essbase uses a single bitmap if there is more than one anchoring dimension or if the calculator cache is not large enough to support multiple bitmaps, and uses two or more bitmaps if there is a single anchoring dimension.
A single bitmap has these properties:
Multiple bitmaps have these properties:
Essbase chooses one of three options for the calculation:
Option |
Method |
Performance Rating |
---|---|---|
Essbase chooses the optimal performance method for a database calculation, based on the size of the calculator cache. If the calculator cache size is too small for any of the above options, Essbase does not use a calculator cache. Calculation performance may be significantly impaired.
Enabling parallel calculation may change the which calculator cache option is used. See Calculator Cache for details.
Caution: If you are calculating the database for the first time, the size of the calculator cache is particularly significant for calculation performance. If possible, ensure that the calculator cache is large enough for Essbase to use the optimal calculator cache option.
The optimum size of the calculator cache depends on the amount of memory the system has available. It also depends on the nature and configuration of the database. You can calculate the calculator cache size that is required for each of the three options in Table 82.
Consider an example database with five sparse dimensions (S1 to S5):
Sparse Dimension |
# of Members |
Dependent Parents |
---|---|---|
Use this example information for these sample calculations:
Option 1: Single Anchoring Dimension, Multiple Bitmaps
For this example calculation, assume the following facts about a database (from Table 82):
Maximum number of dependent parents in the anchoring dimension | ||
In order for Essbase to use multiple bitmaps for this database with a single anchoring dimension, the calculator cache needs to be 625,000 bytes.
Option 2: Single Anchoring Dimension, Single Bitmap
For this example calculation, assume the following facts about a database (from Table 82):
In order for Essbase to use a single bitmap for this database with a single anchoring dimension, the calculator cache needs to be 125,000 bytes.
Option 3: Multiple Anchoring Dimensions, Single Bitmap
For this example calculation, assume the following facts about a database (from Table 82):
In order for Essbase to use a single bitmap for this database with a single anchoring dimension, the calculator cache needs to be 2,500 bytes.
Choosing a Calculator Cache Size for a Database
The following table shows which calculator cache option Essbase uses, depending on the calculator cache size specified:
Minimum Size Specified |
Option Selected |
If you specify a calculator cache size of less than 2,500 bytes, Essbase does not use a calculator cache during the calculation. Calculation performance may be significantly impaired.
You can check which calculator cache option Essbase is able to use on a database by using the SET MSG SUMMARY command in a calc script. Run the following calc script on the empty database:
SET MSG SUMMARY; CALC ALL;
Essbase displays the calculator cache setting in the ESSCMD window or in the application log. For more information, see Monitoring and Tracing Calculations.
The maximum calculator cache size that you can specify is 200,000,000 bytes. The default is 200,000 bytes. The calculator cache size that you choose depends on how much memory is available and the configuration of the database.
Note: The sizes of the calculator, index, data file, and data caches usually have a greater effect on performance if the database calculation is based more on aggregations and less on formula calculations.
If you are calculating the database for the first time, the size of the calculator cache is particularly significant. If possible, ensure that the calculator cache is large enough for Essbase to use the optimal calculator cache option. For more information, see Calculating the Calculator Cache Size.
You can use the default calculator cache size, or you can set the size of the calculator cache within a calc script. If you set the size from a calc script, the setting is used only for the duration of the calc script. For more information, review information about the calc script SET CACHE command and the CALCCACHE configuration setting in the Technical Reference in the docs directory.
You can use the SET CALCHASHTBL command to optimize how Essbase uses the calculator cache when it calculates large, flat database outlines (for example, where one member has more than 5000 children). To use SET CALCHASHTBL, you must enable the calculator cache.
When you enable SET CALCHASHTBL, Essbase uses a hash table to optimize use of the calculator cache for large, flat databases. Using this feature may significantly improve the performance of a CALC ALL of the database or a CALC DIM of the dimension containing the member with over 5000 children.
For more information, see the SET CALCHASHTBL command and the CALCOPTCALCHASHTBL and CALCHASHTBLMEMORY configuration settings in the Technical Reference in the docs directory.
Hyperion Essbase uses a separate dynamic calculator cache for each open database. The DYNCALCCACHEMAXSIZE setting in the configuration file, ESSBASE.CFG, specifies the maximum size of each dynamic calculator cache on the server. By default, the maximum size is 20 MB. Hyperion Essbase allocates area in a dynamic calculator cache for data blocks until the maximum memory area specified by the DYNCALCACHEMAXSIZE setting is allocated.
For more information about DYNCALCACHEMAXSIZE and other dynamic calculator cache settings, see Changing the Dynamic Calculator Cache Size.
For each database, Essbase writes two messages to the application log for each data retrieval:
The first message describes the total amount of time required for the retrieval. If a dynamic calculator cache is used, the second message displays the number of blocks calculated within the data calculator cache (DCC = n) and the number of blocks calculated in general memory (non-DCC = n).
Four configuration file settings are relevant to dynamic calculator caches. The optimum values for these four dynamic calculator cache settings depend on the amount of memory on the server machine, the configuration of all databases on the server machine, and the nature of user queries.
Table 83 describes each setting and includes recommendations on how to determine values for your system. To match your site's unique requirements, you may need to test and adjust the settings.
DYNCALCCACHEMAXSIZE |
|
This setting specifies the maximum size Hyperion Essbase can allocate to each dynamic calculator cache on the server. |
|
Recommended setting value = C * S * U.
For example, for the dense dimensions in Sample Basic, 12 (Year) * 8 (Measures) * 3 (Scenario) * 8 bytes = 2304 bytes. Assigning the value 0 (zero) to DYNCALCACHEMAXSIZE tells Hyperion Essbase not to use dynamic calculator caches. |
|
DYNCALCCACHEWAITFORBLK |
|
If Essbase uses all of the area allocated for a dynamic calculator cache, this setting tells Essbase whether to wait until space becomes available in the cache or to immediately write and calculate the blocks in memory outside the dynamic calculator cache. If the dynamic calculator cache is too small, it is possible for more than one thread to be in queue, each thread waiting to calculate its data blocks. |
|
DYNCALCCACHEBLKTIMEOUT |
|
If Essbase is to wait for available space in the dynamic calculator cache, this setting defines how long it waits. |
|
Recommended setting value = WT / B.
To determine the value of B, check the messages in the application log for the largest number of Dyn.Calc.Cache "Big Block Allocs" for a query, as shown in Figure 416. |
|
DYNCALCCACHEBLKRELEASE |
|
If Essbase has waited the specified time and space is still not available in the dynamic calculator cache, this setting tells Essbase whether to write and calculate the blocks immediately outside the dynamic calculator cache or to create space in the dynamic calculator cache by swapping out blocks and temporarily compressing the swapped blocks in a dynamic calculator cache compressed-block buffer. |
|
Recommended setting value = FALSE (default value). Set to TRUE only if you are experiencing severe memory shortage problems. |
|
DYNCALCCACHECOMPRBLKBUFSIZE |
|
If Essbase has waited the specified wait time and the DYNCALCCACHEBLKRELEASE setting is TRUE, this setting is the size of the dynamic calculator cache compressed-block buffer. |
|
Recommended setting value = (C * S) / 2.
|
Note: After changing any parameter in the essbase.cfg file, you must close and restart Essbase to use the new values.
For more information about dynamic calculator cache settings, see the Technical Reference in the docs directory.
After using a database at your site with typical data, user access, and standard environment (including server machines, network, etc.), check to see how Essbase performs. It is difficult to predict optimal cache sizes without testing. You may need to adjust cache settings.
The sizes of the index cache and the data file cache (when direct I/O is used) are the most critical Essbase cache settings. In general, the larger these caches, the less swapping activity occurs; however, it does not always help performance to set cache sizes larger and larger. Read this entire section to understand cache size considerations.
The advantages of a large index cache start to level off after a certain point. Whenever the index cache size equals or exceeds the index size (including all index files on all volumes), performance does not improve. However, to account for future growth of the index, you can set the index cache size larger than the current index size. See Index Files for an example of estimating index size.
If possible, set the data file cache to equal the size of the stored data, which is the combined size of all ESS*.PAG files. Otherwise, the data file cache should be as large as possible. If you want to account for future growth of stored data, you can set the data file cache size larger than the current size of stored data.
Note: The data file cache is used only if you are using direct I/O.
The data cache should be about 0.125 times the data file cache. However, certain calculations require a larger data cache size. If many concurrent users are accessing different data blocks, this cache should be larger.
In general, if you have to choose between allocating memory to the data file cache or allocating it to the data cache, choose the data file cache if you are using direct I/O. If you are migrating from a previous version of Essbase, see the Essbase Installation Guide for important migration information.
You can check Essbase performance in several ways. One way is to check the Run-time page of the Database Information dialog box. A second method is to click the Statistics tab on the Database Properties window in Administration Services.
To check values in the Run-time page of the Database Information dialog box, select Database > Information from the Application Manager menu, and check the hit ratios:
The sizes of the index cache and the data file cache are the most critical Essbase cache settings. In general, the larger these buffers, the less swapping activity occurs; however, it does not always help performance to set cache sizes larger and larger. Read this entire section to understand the cache size considerations.
The advantages of a large index cache start to level off after a certain point. At any given time, when the index cache size equals or exceeds the index size (including all index files on all volumes), performance does not improve. However, to account for future growth of the index, you can set the index cache size larger than the current index size.
Because the index cache is filled with index pages, for optimum use of storage, set the size of the index cache to be a multiple of the size of the index page (8 KB).
If possible, set the data file cache to equal the size of the stored data, which is the combined size of all ESS*.PAG files. Otherwise, the data file cache should be as large as possible. If you want to account for future growth of stored data, you can set the data file cache size larger than the current size of stored data.
The data cache should be about 0.125 times the data file cache. However, certain calculations require a larger data cache size. If many concurrent users are accessing different data blocks, this cache should be larger.
Note: The data file cache is used only if you are using direct I/O.
In general, if you have to choose between allocating memory to the data file cache or allocating it to the data cache, choose the data file cache. If you are migrating from a previous version of Essbase, see the Essbase Installation Guide for important migration information.
You can check cache statistics for a database by using the GETPERFSTATS command in ESSCMD. For more information on the GETPERFSTATS command, see the Technical Reference in the docs directory.
Monitoring Performance, provides more information about ways to check performance.
Because calculations are the most processor-intensive operations on a Essbase database, you should run test calculations and examine how various cache sizes affect memory use on OLAP Server.
![]() © 2002 Hyperion Solutions Corporation. All rights reserved. http://www.hyperion.com |