global cache cr request

When a session is looking for a consistent read (cr) copy of the buffer cached by the remote instance it waits on the global cache cr request event until the buffer is available in the local instance. Normally the holder constructs the CR version of the buffer and ships it to the requesting instance through the high speed interconnect. The wait ends when the CR buffer arrives at the local buffer cache.

Depending on the mode of the buffer held in the remote instance and the number of times the remote instance ships the CR copy to the local instance one of the following will happen:

1. The CR copy of the buffer is sent to the requesting instance and the 'fairness counter' is incremented for that buffer. Subsequently the 'global cache cr blocks received' statistics is incremented.
2. If the buffer is not cached by the remote instance (but mastered by the remote instance) it grants it to the requesting instance. The session can read the block from the disk and the global cache gets statistics is incremented.
3. If the number of CR copies for that particular buffer has reached the _fairness_threshold limit, the remote instance just downgrades the lock to null mode and flushes the redo to the disk. The session can do the disk I/O to get the block from the disk.

In cases b) and c) the global cache cr requests waits are typically followed by the db file sequential reads and/or db file scattered read waits.

When a process requires one or more blocks, Oracle first checks whether we have those (blocks) in its local cache. The simple hashing algorithm based on the DBA (Data Block Address) is used to traverse the cash buffer chain latches and the suitable lock is found if that is in the hash bucket. When a session requests a block(s) that was not found in its local cache, it will request the resource master to grant the shared access to those blocks. If the blocks are in the remote cache, then the blocks are transferred using the interconnect to the local cache. The time waited to get the blocks from the remote cache is accounted in global cache cr request wait event. NOTE: This event is known as 'gc cr request' in Oracle 10g and all 'global cache' waits are prefixed as 'gc' waits.

The time to get the buffer from remote instance to local instance depends on whether the buffer is in shared mode or exclusive mode. If the buffer is in shared mode the remote instance will just clone the buffer to the local instance and ship it to the local cache. Based on the fairness value on that buffer, the lock downgrade may also happen if the number of cr blocks exceeds the _fairness_threshold counter value. If the buffer is in exclusive mode (XCUR) then, the PI has to be built and the PI is shipped across the buffer cache and the statistics are incremented depending whether the CUR block is shipped or CR block is shipped.

Normally global cache cr request waits are typically followed by the db file sequential/scattered read waits. During the sequential scan there are chances that a few blocks may be in the remote buffer cache and the rest of the blocks are on the disks. Normally the requesting process will wait up to 100cs and retry reading the same block either from the disk or wait for the buffer from remote cache depending on the status of the lock. Excessive waits for global cache cr request may be an indication of a slow interconnect. The private interconnect (aka high speed interconnect) should be used for cache transfer between instances and the public networks should be used for client server traffic. In some cases, the RAC may not rightly pick the interconnect and the cache fusion traffic may be routed through the public network. In this case you will see the huge number of waits for global cache cr request. We can use the ORADEBUG IPC command to verify if the private network is used for cache transfer between instances.

Finding the interconnect used for Cache Transfer

The following procedure can be used to find the interconnect used for cache fusion.

SQL> oradebug setmypid
Statement processed.
SQL> oradebug ipc
Information written to trace file.
SQL> oradebug tracefile_name
/oracle/app/oracle/product/9.2.0/admin/V920/udump/v9201_ora_16418.trc
SQL>

The trace file will contain the details of the IPC information along with the interconnect details.

SKGXPCTX: 0xad95d70 ctx
admono 0xbcfc2b9 admport:
SSKGXPT 0xad95e58 flags         info for network 0
        socket no 8     IP 192.168.0.5  UDP 38206
        sflags SSKGXPT_UP
        info for network 1
        socket no 0     IP 0.0.0.0      UDP 0
        sflags SSKGXPT_DOWN
        active 0        actcnt 1
context timestamp 0

From the above trace file we know the private network 192.168.0.5 is used for the cache fusion traffic and the protocol used is UDP. NOTE: For some reasons if the right interconnect is not used by the Oracle kernel, the cluster_interconnects parameter can be used to specify the interconnect for cache fusion. But this limits the fail over capability during interconnect failures.

Wait Parameters:

• P1 = File number from which Oracle is reading the block from
• P2 = Block number from the file that Oracle wants to read into a buffer file#
• P3 = The lock element number or class of the buffer

The following table lists the most commonly seen block classes:

Wait Time:

The wait is up to 1 second between timeouts. The actual wait time will be until the buffer is cached at the requesting instance in required compatible mode.

The statistics related to global cache are available from V$sysstat. The following SQL provides the details about the global cache statistics:

REM In Oracle10g global cache waits are known as gc waits
REM Replace 'global cache%' with 'gc%' for Oracle10g
SELECT NAME
from V$SYSSTAT
where NAME like '%global cache%';

global cache getsglobal cache get time

global cache converts

global cache convert time

global cache cr blocks received

global cache cr block receive time

global cache current blocks received

global cache current block receive time

global cache cr blocks served

global cache cr block build time

global cache cr block flush time

global cache cr block send time

global cache current blocks served

global cache current block pin time

global cache current block flush time

global cache current block send time

global cache freelist waits

global cache defers

global cache convert timeouts

global cache blocks lost

global cache claim blocks lost

global cache blocks corrupt

global cache prepare failures

global cache skip prepare failures